Re: [akka-user] [Akka-HTTP] Request for comment - Post render regex modifier
Crap, sorry Konrad. Will do. On Monday, September 19, 2016 at 2:53:59 PM UTC-4, Konrad Malawski wrote: > > Hi Derek, > could you re-post this on github.com/akka/akka-http? > I want to have a look but having a very chaotic week - it'd help me/us to > have it as ticket - easier to find and come back to it than on akka-user. > > I think we should be able to help in some way or another if servers are > doing "weird stuff", > I have not yet reviewed your proposal though, so let's see. > > Thanks for attempting to solve it in Akka itself, should be of good > benefit to everyone :) > > -- > Konrad `ktoso` Malawski > Akka <http://akka.io> @ Lightbend <http://lightbend.com> > > On 19 September 2016 at 11:46:58, Derek Wyatt (de...@derekwyatt.org > ) wrote: > > Hi folks, > > I've been wrestling with a bug in IIS (see > https://groups.google.com/d/msg/akka-user/7iW5-HlshZk/S75Wgxg_AwAJ). It > seems that I'm going to have to deal with the problem myself, because > either IIS can't be fixed or the guys I'm dealing with aren't willing to > try. And, at any rate, since I've used Akka HTTP to build a Reverse HTTP > Proxy, I'm sure I'm going to have to deal with oddities as part of every > day life so an enhancement as the one I'm proposing is vital to my success. > > As such, I've thrown together a simple (and highly inelegant at the > moment) mechanism for being able to rewrite the Content-Type header after > rendering, as this is the header that is currently causing me problems. > > I'd very much appreciate someone taking a look at it and critique the > approach. I tried a number of ways of shoehorning this feature into the > code, implicits on the render method, a template-method pattern for wiring > in a user-defined transformer and they were all pretty cumbersome. This > was the "simplest" method I could find for the moment. > > I don't have any default values for the arguments because I wanted to make > sure I was able to write it up properly all the way through. > > Does anyone think this idea (once cleaned and made "proper") could be > accepted as a PR? I'm still working on it a fair bit because I've got to > get something into production that will allow me to work around the IIS > problem, but I expect the essence is there. > > > https://github.com/derekwyatt/akka/commit/1b279bc426be352a1d204e45e855451acaac58e3 > > Thanks, > Derek > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+...@googlegroups.com . > To post to this group, send email to akka...@googlegroups.com > . > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] [Akka-HTTP] Request for comment - Post render regex modifier
Hi folks, I've been wrestling with a bug in IIS (see https://groups.google.com/d/msg/akka-user/7iW5-HlshZk/S75Wgxg_AwAJ). It seems that I'm going to have to deal with the problem myself, because either IIS can't be fixed or the guys I'm dealing with aren't willing to try. And, at any rate, since I've used Akka HTTP to build a Reverse HTTP Proxy, I'm sure I'm going to have to deal with oddities as part of every day life so an enhancement as the one I'm proposing is vital to my success. As such, I've thrown together a simple (and highly inelegant at the moment) mechanism for being able to rewrite the Content-Type header after rendering, as this is the header that is currently causing me problems. I'd very much appreciate someone taking a look at it and critique the approach. I tried a number of ways of shoehorning this feature into the code, implicits on the render method, a template-method pattern for wiring in a user-defined transformer and they were all pretty cumbersome. This was the "simplest" method I could find for the moment. I don't have any default values for the arguments because I wanted to make sure I was able to write it up properly all the way through. Does anyone think this idea (once cleaned and made "proper") could be accepted as a PR? I'm still working on it a fair bit because I've got to get something into production that will allow me to work around the IIS problem, but I expect the essence is there. https://github.com/derekwyatt/akka/commit/1b279bc426be352a1d204e45e855451acaac58e3 Thanks, Derek -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Akka-HTTP] How do I not specify the charset?
We have been able to see it work, if we put the charset *between* the multipart/form-data and the boundary. i.e. Content-Type: multipart/form-data; charset=utf-8; boundary=some-nonsense-here Which should reasonably work for everyone, including this busted app. I know that it messes with the rendering a little bit (i.e. the MediaType is rendered, and then the charset is rendered), but would you guys consider a PR with that change? On Tuesday, September 13, 2016 at 1:37:39 PM UTC-4, Derek Wyatt wrote: > > I'm not arguing that Akka is doing the wrong thing. > > But, I'm pretty screwed right now because significant functionality of the > app for which I'm proxying is broken due to this and it's probably going to > be my problem to fix :) So, is there any way that I can hook into the > parsing and rendering of the request in order to munge it into something > that doesn't break this poorly behaving app? > > On Tuesday, September 13, 2016 at 1:29:03 PM UTC-4, √ wrote: >> >> "The "charset" parameter is used with some media types to define the >> character set (section 3.4) of the data. When no explicit charset parameter >> is provided by the sender, media subtypes of the "text" type are defined to >> have a default charset value of "ISO-8859-1" when received via HTTP. Data >> in character sets other than "ISO-8859-1" or its subsets MUST be labeled >> with an appropriate charset value. See section 3.4.1 >> <https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.4.1> for >> compatibility problems." >> >> https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.7 >> >> -- >> Cheers, >> √ >> >> On Sep 13, 2016 18:58, "Derek Wyatt" <de...@derekwyatt.org> wrote: >> >>> Hi guys, >>> >>> I've written a reverse-proxy with Akka-HTTP and I'm hitting a problem >>> while trying to interact with an ASP.NET app. The only interesting >>> thing I can see is that without the proxy, the Content-Type is lacking a >>> charset but the proxy adds charset=UTF-8. >>> >>> I can't find a way to replicate that using Akka HTTP. I think we can >>> all agree that the addition of charset=UTF-8 is the right thing to do, but >>> in this case it may be causing us a problem and I'd like to force it to >>> look the same. The best I've gotten so far is a definition where it has >>> 'charset=', which isn't quite right. >>> >>> Can this be done? >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [Akka-HTTP] How do I not specify the charset?
I'm not arguing that Akka is doing the wrong thing. But, I'm pretty screwed right now because significant functionality of the app for which I'm proxying is broken due to this and it's probably going to be my problem to fix :) So, is there any way that I can hook into the parsing and rendering of the request in order to munge it into something that doesn't break this poorly behaving app? On Tuesday, September 13, 2016 at 1:29:03 PM UTC-4, √ wrote: > > "The "charset" parameter is used with some media types to define the > character set (section 3.4) of the data. When no explicit charset parameter > is provided by the sender, media subtypes of the "text" type are defined to > have a default charset value of "ISO-8859-1" when received via HTTP. Data > in character sets other than "ISO-8859-1" or its subsets MUST be labeled > with an appropriate charset value. See section 3.4.1 > <https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.4.1> for > compatibility problems." > > https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.7 > > -- > Cheers, > √ > > On Sep 13, 2016 18:58, "Derek Wyatt" <de...@derekwyatt.org > > wrote: > >> Hi guys, >> >> I've written a reverse-proxy with Akka-HTTP and I'm hitting a problem >> while trying to interact with an ASP.NET app. The only interesting >> thing I can see is that without the proxy, the Content-Type is lacking a >> charset but the proxy adds charset=UTF-8. >> >> I can't find a way to replicate that using Akka HTTP. I think we can all >> agree that the addition of charset=UTF-8 is the right thing to do, but in >> this case it may be causing us a problem and I'd like to force it to look >> the same. The best I've gotten so far is a definition where it has >> 'charset=', which isn't quite right. >> >> Can this be done? >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+...@googlegroups.com . >> To post to this group, send email to akka...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Re: [Akka-HTTP] How do I not specify the charset?
I was able to verify that this is breaking the app in question, but not with Akka. I merely added the charset to the request that was working, and it broke. But I think I have a more fundamental problem: I can't see the raw request that came in from the client. It, of course, is only represented using the Akka HTTP model and that model requires a charset for NonBinary ContentTypes. So, if I wanted to try and force the outgoing request (to the server for which I am being a reverse-proxy) to match the incoming request in this regard, I wouldn't have enough information to know that I should do that. Is there any way to get at the raw HTTP request so I can inspect it? I think I need to see exactly how the client created it in the first place. On Tuesday, September 13, 2016 at 12:58:15 PM UTC-4, Derek Wyatt wrote: > > Hi guys, > > I've written a reverse-proxy with Akka-HTTP and I'm hitting a problem > while trying to interact with an ASP.NET app. The only interesting thing > I can see is that without the proxy, the Content-Type is lacking a charset > but the proxy adds charset=UTF-8. > > I can't find a way to replicate that using Akka HTTP. I think we can all > agree that the addition of charset=UTF-8 is the right thing to do, but in > this case it may be causing us a problem and I'd like to force it to look > the same. The best I've gotten so far is a definition where it has > 'charset=', which isn't quite right. > > Can this be done? > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] [Akka-HTTP] How do I not specify the charset?
Hi guys, I've written a reverse-proxy with Akka-HTTP and I'm hitting a problem while trying to interact with an ASP.NET app. The only interesting thing I can see is that without the proxy, the Content-Type is lacking a charset but the proxy adds charset=UTF-8. I can't find a way to replicate that using Akka HTTP. I think we can all agree that the addition of charset=UTF-8 is the right thing to do, but in this case it may be causing us a problem and I'd like to force it to look the same. The best I've gotten so far is a definition where it has 'charset=', which isn't quite right. Can this be done? -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] [Akka-HTTP] How do I drop the connection from the Server?
Hi guys, A client connects, and I don't like him. I don't want to respond to him at all; I just want to disconnect him entirely. I'm using the high level routing DSL and I've read some things about cancelling the HTTPResponse Entity's stream, but that doesn't seem to be dropping the connection. Can this be done using the high level API or do I need to refactor my code to use the low level API? Thanks, Derek -- >> Read the docs: http://akka.io/docs/ >> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Stream Testkit expectNextType ?
Alright. I'll play around with these over the next few days and send you a
PR.
On Fri, Apr 8, 2016 at 12:12 PM Patrik Nordwall <patrik.nordw...@gmail.com>
wrote:
> Hi Derek,
> Sounds like a great addition. Will you take a stab at it?
> /Patrik
>
> On Thu, Apr 7, 2016 at 9:59 PM, Derek Wyatt <de...@derekwyatt.org> wrote:
>
>> Hi guys,
>>
>> I've been writing some tests for my stream code and have run up against a
>> situation where I would like to have expectNextType. I've implemented
>> this myself with a pimp, but I'm wondering if there's a better way?
>>
>> implicit class ProbeExtension[T](val p: Probe[T]) {
>> def expectNextType[A](implicit t: ClassTag[A]): p.Self = {
>> p.expectNextPF {
>> case a: A =>
>> case x => fail(s"Expected a $t type but got a
>> ${x.getClass.getName}")
>> }
>> p
>> }
>> }
>>
>> This is mostly useful when the 'A' in question is a derivation of Try[_].
>> I tend to write expectNextType[Failure[_]] in these cases. And, just to
>> give some context as to why I would want to do that: My flow components
>> might return a Try, which I then pass through a special handling / logging
>> flow, which transforms them to a List that is composed with a mapConcat.
>> So...
>>
>> val flowReturningTry: Flow[T, Try[U], NotUsed] = ...
>> val handlingAndLogging: Flow[Try[U], U, NotUsed] = Flow[Try[U]].mapConcat
>> { /* List(u) or List(), depending on the Try */ }
>> val composed: Flow[T, U, NotUsed] =
>> flowReturningTry.mapConcat(handlingAndLogging)
>>
>> Now, when testing flowReturningTry in isolation, the expectNextType is
>> useful.
>>
>> Cheers and thanks,
>> Derek
>>
> --
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> ---
>>
> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+unsubscr...@googlegroups.com.
>
>
>> To post to this group, send email to akka-user@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
>
> Patrik Nordwall
> Akka Tech Lead
> Lightbend <http://www.lightbend.com/> - Reactive apps on the JVM
> Twitter: @patriknw
>
> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ:
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Akka User List" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/akka-user/ySk9lP-TZQU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] Stream Testkit expectNextType ?
Hi guys,
I've been writing some tests for my stream code and have run up against a
situation where I would like to have expectNextType. I've implemented this
myself with a pimp, but I'm wondering if there's a better way?
implicit class ProbeExtension[T](val p: Probe[T]) {
def expectNextType[A](implicit t: ClassTag[A]): p.Self = {
p.expectNextPF {
case a: A =>
case x => fail(s"Expected a $t type but got a
${x.getClass.getName}")
}
p
}
}
This is mostly useful when the 'A' in question is a derivation of Try[_].
I tend to write expectNextType[Failure[_]] in these cases. And, just to
give some context as to why I would want to do that: My flow components
might return a Try, which I then pass through a special handling / logging
flow, which transforms them to a List that is composed with a mapConcat.
So...
val flowReturningTry: Flow[T, Try[U], NotUsed] = ...
val handlingAndLogging: Flow[Try[U], U, NotUsed] = Flow[Try[U]].mapConcat {
/* List(u) or List(), depending on the Try */ }
val composed: Flow[T, U, NotUsed] =
flowReturningTry.mapConcat(handlingAndLogging)
Now, when testing flowReturningTry in isolation, the expectNextType is
useful.
Cheers and thanks,
Derek
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] [akka-streams] Consuming on TCP server side - is this really the best way?
Hey Endre,
Thanks so much for doing this. I've looked through it (and skimmed the
new doc page) but haven't digested it well enough to say anything other
than thanks. :) I'll get back to you in the next day or so, perhaps
with a couple of questions.
Akka Team wrote:
And now I added another version where the server just streams random
numbers until the client disconnects, then it closes the connection.
It needed a custom stage though to make emitting from an Iterable
interruptible (mapConcat does not interrupt on completion, only on
errors).
On Wed, Jul 29, 2015 at 1:59 PM, Endre Varga endre.va...@typesafe.com
mailto:endre.va...@typesafe.com wrote:
I now updated the gist with the reverse direction: Now a client
sends a String command and expects an Iterable[Int] back as a
response. I currently limited the funcionality to one request per
connection, since otherwise I would need a bit more elaborate
codec which would complicate the example (I would need to add a
delimiter between the iterables on the wire. Not too hard to add
it though). It still shows how these things are supposed to work.
-Endre
On Wed, Jul 29, 2015 at 1:14 PM, Akka Team
akka.offic...@gmail.com mailto:akka.offic...@gmail.com wrote:
Hi Derek,
It is not that hard, but you need to develop a certain kind of
intuition to attack these problems. I very much recommend the
new documentation page
http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/scala/stream-composition.html
as it helps to visualize the ideas.
I created a sample app that does what you want, you can find
the gist here: https://gist.github.com/drewhk/25bf7472db04b5699b80
The features in that app:
- exposes the client API as a Source[Int, Unit]. Anytime you
materialize that source and send it data, it will open a TCP
connection and dump the integers to the server, then closes
the connection
- exposes the server API as a Source[(InetSocketAddress,
Iterable[Int]), Future[ServerBinding]]. It will provide you
with a continuous stream of client address, client data
iterable pairs.
- includes a simple codec pair for encoding the Ints. It is
kind of stupid for this use case, but it works.
Some notes:
- draining the client data to an Iterable might be suboptimal
if the Iterables are large, in this case a Source[Int] would
be a better abstraction
- the implementation caps the size of the Iterable but
currently just silently ignores overflows (I was lazy to build
a stage or use fold for this sample, so I used grouped())
-Endre
On Sun, Jul 26, 2015 at 9:12 PM, Derek Wyatt
de...@derekwyatt.org mailto:de...@derekwyatt.org wrote:
Hi,
I'm still trying to figure out the best way to work with
TCP flows and, while I've got something working, this
seems really quite wrong, so there's gotta be a better way.
What I want to do is send an Iterable[Int] from the client
to the server and have the server materialize that
resulting flow in a Future[Iterable[Int]].
||
val bytesStage =// elided... BidiFlow of serialization and
framing
val serverValuePromise =Promise[Seq[AnyRef]]()
// Technically, the materialized value isn't important,
since it's actually going to be pulled out
// via the Promise
val
serverConsumerFlow:Flow[AnyRef,AnyRef,Future[Seq[AnyRef]]]=Flow.wrap(
// Consume the client's stream and complete the
serverValuePromise with its folded result
Sink.fold(Vector.empty[AnyRef])((acc,v:AnyRef)=acc
:+v).mapMaterializedValue(v
={serverValuePromise.completeWith(v);v }),
// We're not sending anything from this side
Source.empty)(Keep.left)
// The server
val
serverSide:Future[ServerBinding]=StreamTcp().bindAndHandle(serverConsumerFlow.join(bytesStage),0.0.0.0,0,halfClose
=true)
// We really want to stop listening once the client has
successfully connected, but this is good
// enough
serverValuePromise.future.onComplete {
case_ =
serverSide.onSuccess {
casebinding =binding.unbind()
}
}
// I need the endpoint where the client needs to connect
val destination
=Await.result(serverSide,1.second).localAddress
// Get the source running
Source((1to
10).map(newInteger(_))).via(bytesStage.joinMat(StreamTcp().outgoingConnection(destination))(Keep.right)).to(Sink.ignore).run()
// Print out what the client has sent to the server
Await.result(serverValuePromise.future,1.second).foreach(t
=println(stt: $t))
I tried doing this the other way around - where the server
side supplies source - but this caused me issues with
actually shutting down the socket. Having the client do it
seems to make shutting down the socket on completion of
the source, just naturally occur. The problem with the
server side providing the source was that the client
source needed to finish properly. If I created it as
`empty` then it would kill things too quickly. If I then
created it as a n Actor source that just didn't do
anything, I couldn't find a decent way to close it.
There's gotta be a better way to do this, but I'm too much
of a noob to see it. Can anyone improve this code for me?
Thanks,
Derek
[akka-user] [akka-streams] Consuming on TCP server side - is this really the best way?
Hi,
I'm still trying to figure out the best way to work with TCP flows and,
while I've got something working, this seems really quite wrong, so there's
gotta be a better way.
What I want to do is send an Iterable[Int] from the client to the server
and have the server materialize that resulting flow in a
Future[Iterable[Int]].
val bytesStage = // elided... BidiFlow of serialization and framing
val serverValuePromise = Promise[Seq[AnyRef]]()
// Technically, the materialized value isn't important, since it's actually
going to be pulled out
// via the Promise
val serverConsumerFlow: Flow[AnyRef, AnyRef, Future[Seq[AnyRef]]] = Flow.
wrap(
// Consume the client's stream and complete the serverValuePromise with
its folded result
Sink.fold(Vector.empty[AnyRef])((acc, v: AnyRef) = acc :+ v).
mapMaterializedValue(v = { serverValuePromise.completeWith(v); v }),
// We're not sending anything from this side
Source.empty)(Keep.left)
// The server
val serverSide: Future[ServerBinding] = StreamTcp().bindAndHandle(
serverConsumerFlow.join(bytesStage), 0.0.0.0, 0, halfClose = true)
// We really want to stop listening once the client has successfully
connected, but this is good
// enough
serverValuePromise.future.onComplete {
case _ =
serverSide.onSuccess {
case binding = binding.unbind()
}
}
// I need the endpoint where the client needs to connect
val destination = Await.result(serverSide, 1.second).localAddress
// Get the source running
Source((1 to 10).map(new Integer(_))).via(bytesStage.joinMat(StreamTcp().
outgoingConnection(destination))(Keep.right)).to(Sink.ignore).run()
// Print out what the client has sent to the server
Await.result(serverValuePromise.future, 1.second).foreach(t = println(stt:
$t))
I tried doing this the other way around - where the server side supplies
source - but this caused me issues with actually shutting down the socket.
Having the client do it seems to make shutting down the socket on
completion of the source, just naturally occur. The problem with the
server side providing the source was that the client source needed to
finish properly. If I created it as `empty` then it would kill things
too quickly. If I then created it as a n Actor source that just didn't do
anything, I couldn't find a decent way to close it.
There's gotta be a better way to do this, but I'm too much of a noob to see
it. Can anyone improve this code for me?
Thanks,
Derek
--
Read the docs: http://akka.io/docs/
Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups Akka
User List group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
[akka-user] [Akka-Streams] Dynamic Fan-In and Fan-Out?
Howdie, I'm wondering if it's possible to implement my use case and what the best implementation of that might look like. I'd like to have a bi-directional set of channel messages chunked independently and multiplexed across a TCP connection. Something that looks like this: ChannelMessage(id: Long, msg: Any) -- ChannelByteString(id: Long, bytes: ByteString) -- Seq[ByteString] -- // The ChannelByteString chunked up TCP The chunking and serializing is clear to me and I've got a pipeline for that now. What I'm lost on is the dynamic nature of the channels; the TCP connection is set up ahead of time, and then parties on either end of it negotiate a temporary channel on which they can communicate within it. i.e. there's plexing going on in there. Fan-In and Fan-Out seem to be entirely appropriate to solve this problem, but... I don't want a 1-n or n-1 where n is fixed. I need n to be a variable. What's the best way to do this? Am I looking to using Actors to solve this problem, something more like this: ChannelMessage(id: Long, msg: Any) -- ChannelByteString(id: Long, bytes: ByteString) -- === Actor bridge in here that can install and remove channels === Seq[ByteString] -- // The ChannelByteString chunked up TCP Thanks a lot. It really looks like streams have come a very long way since I last looked. I haven't wrapped my head around them very well as of yet, but I definitely have a feel of the power in them. Bravo :) Cheers, Derek -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Re: How can I reconcile untyped actors with typeful programming?
On 8 March 2014 13:31, Daniel Armak danar...@gmail.com wrote:
Hi Derek.
Yes, I'm aware of the tradeoffs and the reasons for them. It just seems to
me there ought to be a middle road.
Here's an idea off the top of my head. If remoting is going to be
transparent, there won't be compile time type assurance. So let's give up
on local type assurance too, and just use types for documentation and
refactoring; that still seems a lot better than nothing.
Suppose for each actor I wanted to implement, I declare a trait with a
method for each message it might receive. Each method must receive exactly
one parameter, which is the message, and return Unit:
trait Printer {
def print(msg: String): Unit
}
The method name 'print' is just for documentation; only the parameter is
sent as the message. This preserves transparency with the actor model; you
don't rely on the trait implementation to build the actual message.
The user of an ActorRef would use a wrapper (value type around the
ActorRef) generated by a macro to implement this trait. The macro would
also validate the trait: each method has one argument, returns Unit, and
all argument types are distinct. Each method call would be implemented as a
send().
val actor : ActorRef = ???
val typedActor: Printer = mymacro.wrap[Printer](actor) // Or is there a
way to declare types with macros? I forget.
typedActor.print(my message)
The macro would add an (implicit sender: ActorRef) argument to each method.
The actor itself would extend the Printer trait and implement its methods.
Another macro would concatenate them to implement receive:
class PrinterActor extends Actor with Printer with
TypedActorMacro[Printer] {
def print(msg: String):
}
To use become/unbecome, we could introduce something more complicated. Or,
even, use the Printer trait only on the sender side - it would still be
useful. This is just a rough outline. Do you think it might be useful, or
do you think I shouldn't go down this road and trying to marry akka actors
and (actor) types is a futile endeavour?
Not futile, but highly suspect. You've barely scratched the surface with
the above and much research is way ahead of you here.
The ??? you have above isn't exactly trivial to implement, for example.
What's more is that you've probably thrown away a ton more features in the
process. e.g.:
class MyActor(printActor: PrinterActor) { ... }
// later...
val myActor =
MyActor.props(loadBalancer(printerActor.props()))
// oops. Does the loadBalancer now have to
// implement Printer? What if it's a load
// balancer in front of scatter gather routers
// that talk to forwarders that talk to various
// different versions of Printers? Does everyone
// have to implement the same API? If not, how
// are you not throwing away type safety? And, if so
// how am I doing anything but writing annoying code
// that keeps me a slave to the API? And how do I
// easily manage API changes, and so on, and so forth?
Maybe not theoretically futile, but practically? Probably :)
To be perfectly honest, it seems as though you're trying to fix a problem
without having travelled a mile in its shoes yet. There are subsets of the
problem that are much more important and more possible to cage, and you
will see them as you progress. When you do, focusing on those (should you
still believe them to be worth it) might be the far better option.
Thanks!
On Sat, Mar 8, 2014 at 7:51 PM, Derek Wyatt de...@derekwyatt.org wrote:
What you're experiencing is a trade-off. Actors provide a trade-off that
you don't seem to be taking into account; endpoints (Actors) are untyped
and the messages that they handle are strongly typed.
You can't have an Actor be able to process anything with a
type-specific receive method. With Actor programming, I should be able to
add as many intermediaries in the message flow as I like and not disturb
the two endpoints. The intermediaries should equally be ignorant of what's
happening (load balancers, routers, loggers, cachers, mediators,
scatter-gather, and so forth). You should also be able to move them around
a cluster without disturbing the endpoints. You can also set up dynamic
delegation in an Actor without it having to really understand what's going
on - for example, an Actor that speaks the main dialect but delegates to
something else when it doesn't understand what's being said.
If you want to eliminate all of those features, then you will be able to
get the type-safe determinism you're looking for (so long as you stay in
the same JVM - crossing JVMs will incur a what the hell am I *really*
talking
to? question that eliminates a compile time assurance). Roland also tried
bringing the best of both worlds together using Typed Channels but I think
it was lacking a high enough success level to survive, but it might be a
way to get closer to your ideal.
In short, you're giving up type safety in order to open the door
