[akka-user] Re: [ANNOUNCE] Akka 2.4.9-RC1 Released! (Akka HTTP Performance and Entity Streaming)

2016-08-03 Thread Jim Hazen
Some quick feedback on raw iron performance, on my local workstation. Your mileage my vary, but I'm pretty happy with these numbers. Minimal example Webserver code from: http://doc.akka.io/docs/akka/2.4/scala/http/routing-dsl/index.html wrk -d30 -c100 -t12 http://localhost:8080/hello Running

[akka-user] Re: Can I Pause some Flow In the the akka-stream?

2016-02-22 Thread Jim Hazen
Isn't a valve just a specialization of a throttle? Throttle at infinite = open valve Throttle at 0 = closed valve As long as you have an out of band way to adjust this throttle (via materialized value?), you should have your valve already. -- >> Read the docs:

[akka-user] Proper way to shut down an outgoing http request stream?

2016-02-08 Thread Jim Hazen
I have this basic client test. It works way better in 2.4.2-RC2 than ever before, however I still get an error during cleanup: [akka.actor.ActorSystemImpl(HttpClient)] Outgoing request stream error (akka .stream.AbruptTerminationException) Seems like there's a akka-http/streams bug here, or a

Re: [akka-user] Re: Akka HTTP Performance (2.4.2-RC1)

2016-02-08 Thread Jim Hazen
+1 here as well. I've been waiting in anger for akka-http to catch up to spray in terms of performance and have verified some great leaps in recent builds. Thanks guys! If there really is a 6-7x speedup waiting to fall into place (after bug fixing), that would be amazing (and worth the

[akka-user] Re: At present, which one is better to use Akka-HTTP or Spray.io for future based on a new learner?

2015-12-19 Thread Jim Hazen
At present, for a new learner, go with Spray. Spray is mature and fast and all around, "just works". Akka-HTTP is the future of stream based HTTP processing for Akka, but he current 1.0 release lacks the feature parity or performance of Spray. That will hopefully change with the 2.0 release.

Re: [akka-user] Re: CPU-usage and a lot of actors with cluster-systems

2015-11-20 Thread Jim Hazen
But! It is important for me that the work of message 1 is done before > message 2 is handled. And isn't one of the benefits for the actor model and > akka itself, that I have the garantee to that message 1 is done before > message 2? > > Yes. And you might be in trouble there. In which

Re: [akka-user] Re: CPU-usage and a lot of actors with cluster-systems

2015-11-19 Thread Jim Hazen
I'm not saying use a lot of dispatchers. I'm saying that you should delegate to maybe 1 more dispatcher for your heavy work to unblock your actor's dispatching thread (and definitely another for blocking IO) allowing it to put more of your Actor's mailbox entries to work concurrently. In one

Re: [akka-user] Re: Cluster aware consistent hashing pool - works only on leader

2015-11-19 Thread Jim Hazen
Wait, what? So cluster sharding depends on shared mutable state across your cluster to work? AFAIknew the independent local nodes managed their state internally, communicated/coordinated via network protocolling and delegated to a master when it needed to determine a shard owner the first

Re: [akka-user] Re: Cluster aware consistent hashing pool - works only on leader

2015-11-19 Thread Jim Hazen
On Thursday, November 19, 2015 at 12:32:38 PM UTC-8, Patrik Nordwall wrote: > > > > On Thu, Nov 19, 2015 at 8:06 PM, Jim Hazen <jimhaz...@gmail.com > > wrote: > >> Wait, what? So cluster sharding depends on shared mutable state across >> your cluster to w

[akka-user] Re: CPU-usage and a lot of actors with cluster-systems

2015-11-18 Thread Jim Hazen
That's possible. Your actor won't be able to get more work until it completes its receive. You could: A: look into actor pool/routers. This will give you more receive blocks to work with, increasing concurrency. B: look into spending less time within your receive block, maybe by delegating

[akka-user] Re: Cluster aware consistent hashing pool - works only on leader

2015-11-18 Thread Jim Hazen
You don't need your clustered actors to be persistence aware. So you are absolutely free to have sharded stateless actors, or manage state in some other way. The confusing part is that the cluster sharding internals requires akka persistence to be configured for at least one journal. The

Re: [akka-user] [akka-stream] how to control the throughput of a certain flow

2015-11-18 Thread Jim Hazen
Might want to look into mapAsync and mapAsyncUnordered for the linear parts of your slow graph. You can specify the desired parallelism as part of those calls. I was about to write that in general there isn't a "make fast" button for arbitrary graphs. However, that may not be strictly true.

[akka-user] Re: CPU-usage and a lot of actors with cluster-systems

2015-11-17 Thread Jim Hazen
Also, the default akka-remoting utilizing Java serialization is dog slow. There are other threads that discuss swapping out the serializer with much faster ones. So if you're doing a lot of cluster sharding, your throughput may be bottlenecked on the remote inter-node IO. -- >>

Re: [akka-user] Akka-streams TLS-PSK support

2015-11-17 Thread Jim Hazen
Hmm. I'm not an expert here (just use BC for CloudFront). Have you tried keeping your Java 8 SSLContext while combining that with the BC cipher support? Something more like your original, but with the BC ciphers. Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider())

[akka-user] Re: Akka-streams TLS-PSK support

2015-11-16 Thread Jim Hazen
Have you tried adding the BouncyCastleProvider as a security provider? Then making your normal JSSE calls? Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider()) -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >>

[akka-user] Re: Akka Stream and Http 2.0-M1 Released!

2015-11-16 Thread Jim Hazen
With these latest foundational changes, is akka-http 2.0 intended to be the 'spray parity' release? If not, is there a roadmap for an akka-http client as rich as Spray's and akka-http server as performant as Spray's? -- >> Read the docs: http://akka.io/docs/ >> Check

[akka-user] Re: Ask Pattern vs Micro Actors?

2015-08-04 Thread Jim Hazen
http://letitcrash.com/post/96078609472/ask-tell-and-per-request-actors Maybe this will help answer your question. Actors are cheap, but not free. There are cases where using an actor-per-request pattern has more benefits than performance detractors. Per method call? I think that would

Re: [akka-user] memory issue in akka application

2015-08-04 Thread Jim Hazen
If you'd like to handle this in logback land and not deal with altering akka's logging strategy, you can also configure logback to work in bounded space and drop overflows as well. Take a look at logback's AsyncAppender. http://logback.qos.ch/manual/appenders.html You'll configure your

Re: [akka-user] Sharding problem when restarting Cluster

2015-08-04 Thread Jim Hazen
I see this issue happen whenever AWS has a network hiccup. I have a multi-node cluster behind a LB and akka cluster sharding along with akka persistence writing to a Dynamo journal. I'm currently on akka 2.3.11, which means the same shared Dynamo table used to store my persistent actors is

Re: [akka-user] Re: How can I check the number of the entry actors spawned off by akka ShardRegion?

2015-07-20 Thread Jim Hazen
To increase capacity you'll be scaling out nodes, not shard regions. Shards are logical managers/buckets, they help organize and support the actual processing actors, but it's those actors that are distributed. You'll have one for each unique message destination, regardless of your number of

[akka-user] Re: How can I check the number of the entry actors spawned off by akka ShardRegion?

2015-07-20 Thread Jim Hazen
There will be a new actor activated for each unique messageId destination. Messages will arrive at that actor for processing. Actors are grouped into shards, which help manage those actors. If you send 100 messages to unique destinations across your cluster, they'll be handled by 100 unique

Re: [akka-user] Re: How can I check the number of the entry actors spawned off by akka ShardRegion?

2015-07-20 Thread Jim Hazen
, Yifei On Monday, July 20, 2015 at 3:16:24 PM UTC-4, Jim Hazen wrote: There will be a new actor activated for each unique messageId destination. Messages will arrive at that actor for processing. Actors are grouped into shards, which help manage those actors. If you send 100 messages

Re: [akka-user] Akka Streams - How to define a Flow depending on the data coming from the Source? (AKA dinamically)

2015-07-20 Thread Jim Hazen
If you already have the solution it shouldn't be hard to put Streams in front of it to help manage back pressure. Call your code from within the sink and apply whatever flow controls you need to the source ahead of it. -- Read the docs: http://akka.io/docs/ Check the FAQ:

[akka-user] Akka Cluster Performance

2015-07-16 Thread Jim Hazen
Akka clustering is built on top of akka remoting. The default serialization used by remoting is terribly slow. Look into swapping it out with a third party serializer. There are some linked/mentioned in the remoting docs. Please post any results if a serializer change helps you. I know in my

[akka-user] Message Broadcasting in an akka cluster

2015-07-14 Thread Jim Hazen
You could implement the mediator yourself and include all your business logic there. That said, why do this? What's the fear of an extra actor dealing with these concerns and providing separation? I don't think it's possible to use an external implementation and not have separation. --

[akka-user] ShardRegion vs EventBus

2015-07-10 Thread Jim Hazen
Tried looking at the DistributedPubSub module? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are

[akka-user] Best Strategy to Aggregate Multiple Actors Response

2015-06-23 Thread Jim Hazen
Does this work for you? http://doc.akka.io/docs/akka/snapshot/contrib/aggregator.html -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You

Re: [akka-user] 500 ISE for Client Side Error Bug Report

2015-04-16 Thread Jim Hazen
Just because the HTTP protocol doesn't define the semantics doesn't mean that RESTful services aren't allowed to take this on themselves, and some do. Kinda like in the US where rights not reserved for the federal government fall to the states. ElasticSearch is one such example:

Re: [akka-user] How does akka actor helps in achieve concurrency without locking/syncronization

2015-04-16 Thread Jim Hazen
On Wednesday, April 15, 2015 at 9:17:44 PM UTC-7, Rohit Jain wrote: On Wednesday, April 15, 2015 at 10:37:36 PM UTC+5:30, Michael Frank wrote: On 04/15/15 04:47, Rohit Jain wrote: How does using akka actor helps in achieving concurrency without use of any external locks? Is it

[akka-user] Re: 500 ISE for Client Side Error Bug Report

2015-04-15 Thread Jim Hazen
I was originally thinking that this would be inconvenient for ES clients. I could work around that. However for anyone attempting to build a transparent proxy in front of ES (where they can't control client calls), this would be an impossible to fix solution if a GET with body was forbidden.

[akka-user] Re: 500 ISE for Client Side Error Bug Report

2015-04-15 Thread Jim Hazen
Hmm... Spray currently allows this and products like ElasticSearch promote GET bodies. This will be a breaking change for me as well once I migrate to akka-http. How opinionated are the developers here? Can this be more of a guideline than a rule? :) -- Read the docs:

[akka-user] Re: Akka EC2 remoting

2015-04-09 Thread Jim Hazen
Try setting JVM system params. For example, an early demo that I hacked up looked like this. It isn't on EC2, but I don't think that EC2 is really your problem. Hack out the stuff you don't care about and omit the -J if not using sbt. Instance 1: sbt

[akka-user] Akka EC2 remoting

2015-04-06 Thread Jim Hazen
Looks like you're trying to connect to a seed that isn't available. Be sure that your remote.netty.tcp host and port settings match your seed node setting. They don't seem to here. -- Read the docs: http://akka.io/docs/ Check the FAQ:

[akka-user] Re: akka-streams how to chunk ordered stream based on common property efficiently

2015-03-20 Thread Jim Hazen
This should still be easy with akka-streams. You'll probably need to write a custom stage though. Check out the documentation on custom stages along with the cookbook examples. I believe there is an example that implements a batching collector like this. It's not 100% what you're looking

Re: [akka-user] Stream Materialization, a larger discussion.

2015-03-17 Thread Jim Hazen
for your time, Jim On Sunday, March 8, 2015 at 11:24:58 AM UTC-7, rkuhn wrote: Hi Jim, thanks for starting the discussion! Replies inline. 5 mar 2015 kl. 21:30 skrev Jim Hazen jimhaz...@gmail.com javascript:: I've been following Akka Streams for bit. I've noticed the changes in M4

[akka-user] How to use Cluster Sharding, Please do reply it's very urgent

2015-03-16 Thread Jim Hazen
Start here: http://doc.akka.io/docs/akka/2.3.9/contrib/cluster-sharding.html#cluster-sharding -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user ---

[akka-user] How to use Cluster Sharding, Please do reply it's very urgent

2015-03-16 Thread Jim Hazen
Start hers: http://doc.akka.io/docs/akka/2.3.9/contrib/cluster-sharding.html#cluster-sharding -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user ---

[akka-user] Split Stream

2015-03-15 Thread Jim Hazen
Have you looked into writing a custom router component? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are

Re: [akka-user] Re: akka-http: combine path together

2015-03-12 Thread Jim Hazen
I was confused. I thought you were looking for something like: path(v2.0 / tokens) { post { complete(it works) } } ~ path(v2.0 / somethingelse) { post { complete(it works) } } Otherwise with path(v2.0) { path(tokens) { You're matching

Re: [akka-user] Re: Problems using a Remote Actor

2015-03-10 Thread Jim Hazen
Have you followed all of the instructions for interacting with remote actors here: http://doc.akka.io/docs/akka/2.3.9/scala/remoting.html ? From the error it appears to me that: 1. You are attempting to create a remote actor from a peer 2. This peer isn't able to properly interact with

[akka-user] Stream Materialization, a larger discussion.

2015-03-05 Thread Jim Hazen
I've been following Akka Streams for bit. I've noticed the changes in M4 and of course read the materialization sections of: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M4/scala/stream-flows-and-basics.html I have tremendous respect for the Akka team and understand that

[akka-user] Re: Problem with dispatcher configuration

2015-03-05 Thread Jim Hazen
Check the nesting. Does test.my-dispatcher work? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are

Re: [akka-user] Re: (re-)Chunking Source of ByteStrings, filling the gaps

2015-03-05 Thread Jim Hazen
Wouldn't migrating from a grouped to groupedWithin based chunker be the compromise that ppl are looking for? Perform chunk packing within a given window, but if you're waiting for too long, send what you have? You mention here and in the JIRA that this a conflate problem, but conflate deals

Re: [akka-user] [Streams 1.0-M4] Compatibility with Akka-2.4-SNAPSHOT?

2015-03-03 Thread Jim Hazen
Is there an ETA for akka-http on 2.4? According to Mathias, Spray won't be ported to 2.4: https://groups.google.com/d/msg/spray-user/x0KdMn_7exE/B8Rp2xuSa2sJ and according to you akka-http also isn't yet ready for 2.4. I'd like to develop a REST service that takes advantage of cluster sharding

Re: [akka-user] [Streams 1.0-M4] From Source[Future[T], Unit] to Source[T, Unit]?

2015-03-02 Thread Jim Hazen
I think the answer is in the {Scala, Java}Docs of mapAsync: Transform this stream by applying the given function to each of the elements as they pass through this processing step. The function returns a Future and the value of that future will be emitted downstreams. As many futures as

[akka-user] [Streams 1.0-M4] Compatibility with Akka-2.4-SNAPSHOT?

2015-03-02 Thread Jim Hazen
Are these 1.0-M4 modules intended to be compatible with akka-2.4-SNAPSHOT? I'm having trouble passing a custom materializer to an IncommingConnection.handleWithAsyncHandler(). The default materializer seems to work fine, but I wanted to play with buffer sizes. implicit val materializer =

[akka-user] Re: Akka Cluster (and Cluster Client) on Kubernetes

2015-03-02 Thread Jim Hazen
Might want to look at this thread. https://groups.google.com/forum/#!topic/akka-user/wyA4xNfM5LM -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user ---

[akka-user] Re: Akka Cluster (and Cluster Client) on Kubernetes

2015-03-02 Thread Jim Hazen
Lol. Or not, since you're the OP. Oh well, need more coffee. On Monday, March 2, 2015 at 10:52:29 AM UTC-8, Jim Hazen wrote: Might want to look at this thread. https://groups.google.com/forum/#!topic/akka-user/wyA4xNfM5LM -- Read the docs: http://akka.io/docs/ Check the FAQ

[akka-user] Re: akka system shutdown in sub actor

2015-02-26 Thread Jim Hazen
Seems logical. From an actor within a running system you ask the system if it's dead. It says no, because it still has actors running (there's at least the one asking the question). Then, from outside the system you wait for it to terminate, and then check that it has terminated, which it

[akka-user] Re: [Akka-stream] Distributed actor communication via Ask pattern or ActorPublisher/ActorSubscriber

2015-02-21 Thread Jim Hazen
To me, this isn't the correct solution. Passing Source(data) around seems wrong, it would seem dangerous for large files and impossible for remote actors if the source were pointer based. The tl;dr is that IMHO the correct solution, where you process indexing responses in a flow based way,

[akka-user] Re: akka-streams - How to define a Source from an arbitrary event stream?

2015-02-19 Thread Jim Hazen
I'd also be interested to know what the best practice patterns for bridging internal/external stream boundaries are. In this case there isn't yet a well understood pattern for external entities to dynamically feed into a stream source. There is likewise no easy way for an external library to

[akka-user] Re: going from using akka in a play application to scaling the akka to multiple nodes, what to plan for?

2015-02-15 Thread Jim Hazen
Take a look at cluster sharding. By far the easiest way to get started with clustered/distributed actors. http://doc.akka.io/docs/akka/2.3.9/contrib/cluster-sharding.html#cluster-sharding On Sunday, February 15, 2015 at 7:58:02 AM UTC-8, gitted wrote: Currently I am a play application that I

Re: [akka-user] akka-http long-lived connection backpressure problem

2015-02-09 Thread Jim Hazen
Thanks for creating the issue. Please note that there were actually two issues discovered. The client flow issue was logged, but the server request parsing issue wasn't. The rest of this post is really just going through my thought process, so that the group can benefit from an analysis of

Re: [akka-user] akka-http long-lived connection backpressure problem

2015-02-09 Thread Jim Hazen
Now that I think even more on the problem the more it seems that we're trying to solve too much within a single flow. There are concepts that ought to be independent that we're forcing to become dependent. I still like the independent streams model. I'd start with that, but with a

[akka-user] Re: Akka Cluster Actor Failover and Rebalancing, is it possible?

2015-02-07 Thread Jim Hazen
There are probably a few ways to do this. Most straight forward might be to start a local actor as part of each node's startup code. Then use Akka clustering to observe node failures. Use some deterministic strategy to reallocate that actor on some other host (let's say failed +1 in your