Re: [akka-user] Re: Akka Camel and Akka Streams

['Martin Krasser' via Akka User List]

Hi Sam,

not yet, only auto-acks at the moment. I created a ticket for 
application-level acks https://github.com/krasserm/streamz/issues/40. 
Feel free to comment if you think something is missing.


Cheers,
Martin

On 01.08.17 16:21, Sam D wrote:

Martin,

Is there an Ack or Nack/Status.Failure mechanism with the new streamz 
camel api like in the akka-camel actor Consumer api?


Thanks,
Sam

On Thursday, July 13, 2017 at 4:16:52 AM UTC-4, Andreas Gies wrote:

Hi,

thanks for the pointer to the Camel docs. I knew I had a blindspot
somewhere.

Best regards
Andreas

Am Dienstag, 11. Juli 2017 14:26:09 UTC+2 schrieb Andreas Gies:

Hello HAkkers,

we are maintaining an integration framework (OSGi) project
built on top of ActiveMQ, Spray and Camel implemented in Scala [1]

Most of our internal API's rely on Akka and some also on the
Akka-Camel integration.


With the next major release we plan to upgrade our Spray
routes to Akka-Http. Now that I have started looking at the
concrete
steps I have noticed that Akka-Camel is also deprecated and to
be replaced with alpakka.

I had a look through the Alpakka project. So far I haven't
found the replacement for the Akka-Camel efforts in there.
Perhaps I have overlooked something or is the replacement just
on the roadmap ?

Also I did have a look at the JMS part of Alpakka. It seems
that this implementation currently only supports TextMessages
and ignores any properties within the message. If I understand
the implementation correctly, it would fail the stream in case
of any JMSException and also when the buffer has been filled up ?

On a broader level I was wondering if the implementation
should be in the form that the inbound stream fails if and
only if
the connection is irrecoverably broken and in other cases the
Stream should reconnect transparently.

Also on a broader level I have noticed, that the messages are
acknowledged as soon as they pushed. Coming from a JMS
background that feels a bit strange to me, but that might be
because I am unfamiliar with the streaming API. In our world
a message is normally acknowledged when it has been processed
successfully (which is normally it has been written to
the file system, forwarded to another JMS destination or
triggered some execution in the database).

If the container crashes before it has acknowledged the
message, the message will be redelivered. In cases we encounter
an error we pass the message to an error handling destination
or a retry destination.


Apparently the architecture and the acceptance level of
message loss changes when switching to a streaming approach.


For now I have some concrete questions:

1) Have I missed the Camel replacement in Alpakka and if so,
where is it located within Alpakka ?
2) How are others coping with the "window of potential message
loss" when migrating from pure JMS flows to streams ?
3) Any pointers to good hands-on white papers are much
appreciated. I have read through some and also through most
of the streams documentation, but I guess I need to get my
hands dirty ...
4) I don't dare to ask, but if anyone is using Akka / AkkaHttp
and / or AkkaStream within OSGi I would be more than happy
to exchange experiences & ideas.

[1] https://github.com/woq-blended/blended



Thanks in advance for your attention
Best regards
Andreas

--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ: 
http://doc.akka.io/docs/akka/current/additional/faq.html

>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to a topic in the 
Google Groups "Akka User List" group.
To unsubscribe from this topic, visit 
https://groups.google.com/d/topic/akka-user/tl7Gj-nijiw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to 
akka-user+unsubscr...@googlegroups.com 
.
To post to this group, send email to akka-user@googlegroups.com 
.

Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.


--
Martin Krasser

blog:http://krasserm.github.io
code:http://github.com/krasserm
twitter: http://twitter.com/mrt1nz

--

 Read the docs: http://akka.io/docs/
 Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
 Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message 

[akka-user] Re: Akka Camel and Akka Streams

['Martin Krasser' via Akka User List]

Hi Andreas,

the replacement for akka-camel is the Streamz 
 project with its Camel DSL for Akka 
Streams 
. 
It allows you to use any Camel component 
 with Akka Streams for both 
inbound and outbound message exchanges. There is a Scala and Java API. 

Best Regards,
Martin

Am Dienstag, 11. Juli 2017 14:26:09 UTC+2 schrieb Andreas Gies:
>
> Hello HAkkers, 
>
> we are maintaining an integration framework (OSGi) project built on top of 
> ActiveMQ, Spray and Camel implemented in Scala [1]
>
> Most of our internal API's rely on Akka and some also on the Akka-Camel 
> integration. 
>
>
> With the next major release we plan to upgrade our Spray routes to 
> Akka-Http. Now that I have started looking at the concrete 
> steps I have noticed that Akka-Camel is also deprecated and to be replaced 
> with alpakka. 
>
> I had a look through the Alpakka project. So far I haven't found the 
> replacement for the Akka-Camel efforts in there. 
> Perhaps I have overlooked something or is the replacement just on the 
> roadmap ? 
>
> Also I did have a look at the JMS part of Alpakka. It seems that this 
> implementation currently only supports TextMessages 
> and ignores any properties within the message. If I understand the 
> implementation correctly, it would fail the stream in case 
> of any JMSException and also when the buffer has been filled up ?
>
> On a broader level I was wondering if the implementation should be in the 
> form that the inbound stream fails if and only if 
> the connection is irrecoverably broken and in other cases the Stream 
> should reconnect transparently. 
>
> Also on a broader level I have noticed, that the messages are acknowledged 
> as soon as they pushed. Coming from a JMS
> background that feels a bit strange to me, but that might be because I am 
> unfamiliar with the streaming API. In our world 
> a message is normally acknowledged when it has been processed successfully 
> (which is normally it has been written to 
> the file system, forwarded to another JMS destination or triggered some 
> execution in the database). 
>
> If the container crashes before it has acknowledged the message, the 
> message will be redelivered. In cases we encounter 
> an error we pass the message to an error handling destination or a retry 
> destination. 
>
>
> Apparently the architecture and the acceptance level of message loss 
> changes when switching to a streaming approach. 
>
>
> For now I have some concrete questions:
>
> 1) Have I missed the Camel replacement in Alpakka and if so, where is it 
> located within Alpakka ?
> 2) How are others coping with the "window of potential message loss" when 
> migrating from pure JMS flows to streams ?
> 3) Any pointers to good hands-on white papers are much appreciated. I have 
> read through some and also through most 
> of the streams documentation, but I guess I need to get my hands dirty 
> ... 
> 4) I don't dare to ask, but if anyone is using Akka / AkkaHttp and / or 
> AkkaStream within OSGi I would be more than happy 
> to exchange experiences & ideas. 
>
> [1] https://github.com/woq-blended/blended
>
>
> Thanks in advance for your attention 
> Best regards 
> Andreas
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Persistence with Multi-Clusters

['Martin Krasser' via Akka User List]

On 01.02.16 10:27, Patrik Nordwall wrote:
Is the causality tracking (the vector clocks) used for anything else 
than supporting "interactive conflict resolution"? Are they needed 
when using CRDTs?


Operation-based CRDTs as specified in 
http://hal.upmc.fr/file/index/docid/88/filename/techreport.pdf rely 
on a causal delivery order. More generally, for conflict resolution, you 
need to distinguish concurrent updates from causally related ones, 
regardless whether this is done interactively or not. For 
application-defined convergent data types i.e. CRDTs that use custom 
domain logic, they are therefore needed as well.




Based on my (incomplete) understanding of Riak's "failed experiments" 
of exposing vector clocks and letting application/user decide what 
version to continue with I'm skeptical to that approach. I think the 
conflict resolution must be automatic, i.e. CRDTs, and then I wonder 
what the vector clocks adds?




--
Martin Krasser

blog:http://krasserm.github.io
code:http://github.com/krasserm
twitter: http://twitter.com/mrt1nz

--

 Read the docs: http://akka.io/docs/
 Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
 Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Akka Persistence with Multi-Clusters

['Martin Krasser' via Akka User List]

On 31.01.16 17:48, Roland Kuhn wrote:

Hi Martin!

31 jan 2016 kl. 15:36 skrev 'Martin Krasser' via Akka User List 
<akka-user@googlegroups.com <mailto:akka-user@googlegroups.com>>:


Hi Roland,

a few clarifications/additions inline ...

On 31.01.16 10:17, Roland Kuhn wrote:

Hi Paul,

unfortunately it is impossible to make this “just work”—not least 
because you would first have to define what that means. Volker 
mentioned Eventuate as a possible solution, but this also is not 
something that “just works”, it requires your events to be 
structured such that your defined state update functions have all 
the right properties to make it work.


The features and the guarantees provided by Eventuate actually do 
*not* depend on the structure of events. Eventuate provides means to 
distinguish causally related from concurrent events and it is just 
the responsibility of the application to ensure that derived state 
does not depend on the order of concurrent events.


This is the crux of the matter: ensuring that events are commutative 
is not possible in the general case. I am in complete agreement as to 
why ACID (Associative, Commutative, Idempotent, Distributed) makes 
sense, but we should not misrepresent the fact that these systems are 
not trivially equivalent to strongly consistent systems—it will 
usually take quite a bit of thinking  and possibly also adjustments to 
the business requirements to make this work. The result will often be 
desirable for many reasons, but saying that it “just works” is not 
helpful in my opinion; I’m not saying that you implied that, I just 
want to be very explicit about this part.


And so are the Eventuate docs :-) Volker either didn't imply that it 
"just works", he referred to Eventuate as one option how to deal with 
issues that are inevitable when choosing AP over CP. On the other hand, 
these issues are often rather easy to manage, provided an API offers the 
right abstractions for it. That's at least the experience I made from my 
projects in 2015.




Causally related events are always delivered in the same causal order 
at all locations (datacenters, for example). Relaxing strict order to 
causal order is what gives you availability and partition-tolerance 
in multi-datacenter setups.




Imagine there being two copies of yourself running around and doing 
things: it would not be enough for one to tell the other what it has 
done, there can be real conflicts that arise from these independent 
actions (like one of your selves telling your wife that you love her 
and shortly thereafter—without having caught up to that point 
yet—the other files for divorce). The key here is coordination, 
without that certain actions cannot be taken. And coordination can 
be impossible, e.g. due to network partitions, which means that 
you’ll have to decide whether to be cautious or reckless.


Coordination is just an option. If you want to *prevent* concflicts 
(you called it being "cautious"), only then you need coordination. 
Here you give up availability in favor of consistency. The other 
option is to *allow* conflicts, and resolve them later. This does not 
require coordination but rather means to track, detect and resolve 
conflicts. With this option, replicas at different datacenters remain 
writeable, even if they are partitioned from others (you called that 
being "reckless"). However, you "apologize" for being "reckless" with 
conflict resolution :-) A great introductory read on this is Pat 
Helland's paperBuilding on Quicksand 
<http://db.cs.berkeley.edu/cs286/papers/quicksand-cidr2009.pdf>. With 
that second option, you choose availability over (strong) consistency.


Yes, you’re quoting from my recent presentations on the matter :-)

The second option is not only relevant for multi-datacenter 
replication but also more generally for collaboration between 
(micro)services where you usually don't want to couple the 
availability of one service to that of others. The price for that is 
that applications must be able to deal with conflicts, rather than 
trying to prevent them. A consistent approach for that is often 
missing in distributed applications and often solved with error-prone 
ad-hoc solutions. Eventuate tries to change that by not only
providing APIs to track, detect, and resolve conflicts in an 
automated or interactive way but also by providing an infrastructure 
where distributed services can communicate via events in a causally 
consistent and reliable way. The underlying event logs give you the 
following guarantees:


- the order of events in a local event log is consistent with causal 
order i.e. consumers will never see an effect before its cause.
- event replication across different locations is reliable and 
idempotent i.e. consumers will never see duplicates when reading from 
a local log.


These guarantees are certainly valuable, but I am hesitant to declare 
victory jus

Re: [akka-user] Akka Persistence with Multi-Clusters

['Martin Krasser' via Akka User List]

Hi Odd,

glad to hear your interest in such an initiative. I should add that a 
CP/AP use case co-existence is already possible with Eventuate i.e. 
Eventuate can be used like akka-persistence (more or less) and 
additionally allows replication of persistent actors (relaxing strong 
consistency to causal consistency) as well as event collaboration 
between persistent actors. Also, it gives you causal consistency on the 
query side, independent of the storage plugin.


My main intention for combining efforts is the meanwhile rather big 
overlap of Eventuate with akka-persistence. I started Eventuate as a 
separate project to have a sandbox for evaluating my ideas on AP use 
cases. The current approach has meanwhile shown to be useful in several 
applications and is robust for our production environment. So I think 
it's time for combining efforts.


Regards,
Martin

On 31.01.16 16:59, Odd Möller wrote:
Hi Roland and Martin, as a long time happy user of Akka Persistence I 
have for my recent set of projects started looking more and more at 
Martin's work on Eventuate and I really like what I see. The extension 
towards supporting the AP use case as well as the CP use case looks 
really useful. A unification of Akka Persistence and Eventuate that 
would make it possible to build actor based systems where some 
persistence actors supports CP while others supports CP, all using the 
same framework (within a single harmonized API etc) sounds fantastic. 
So +1 on the combining of efforts idea!


Greetings
Odd


On Sun, Jan 31, 2016 at 3:36 PM, 'Martin Krasser' via Akka User List 
<akka-user@googlegroups.com <mailto:akka-user@googlegroups.com>> wrote:


Hi Roland,

a few clarifications/additions inline ...

On 31.01.16 10:17, Roland Kuhn wrote:

Hi Paul,

unfortunately it is impossible to make this “just work”—not least
because you would first have to define what that means. Volker
mentioned Eventuate as a possible solution, but this also is not
something that “just works”, it requires your events to be
structured such that your defined state update functions have all
the right properties to make it work.


The features and the guarantees provided by Eventuate actually do
*not* depend on the structure of events. Eventuate provides means
to distinguish causally related from concurrent events and it is
just the responsibility of the application to ensure that derived
state does not depend on the order of concurrent events. Causally
related events are always delivered in the same causal order at
all locations (datacenters, for example). Relaxing strict order to
causal order is what gives you availability and
partition-tolerance in multi-datacenter setups.



Imagine there being two copies of yourself running around and
doing things: it would not be enough for one to tell the other
what it has done, there can be real conflicts that arise from
these independent actions (like one of your selves telling your
wife that you love her and shortly thereafter—without having
caught up to that point yet—the other files for divorce). The key
here is coordination, without that certain actions cannot be
taken. And coordination can be impossible, e.g. due to network
partitions, which means that you’ll have to decide whether to be
cautious or reckless.


Coordination is just an option. If you want to *prevent*
concflicts (you called it being "cautious"), only then you need
coordination. Here you give up availability in favor of
consistency. The other option is to *allow* conflicts, and resolve
them later. This does not require coordination but rather means to
track, detect and resolve conflicts. With this option, replicas at
different datacenters remain writeable, even if they are
partitioned from others (you called that being "reckless").
However, you "apologize" for being "reckless" with conflict
resolution :-) A great introductory read on this is Pat Helland's
paper Building on Quicksand
<http://db.cs.berkeley.edu/cs286/papers/quicksand-cidr2009.pdf>.
With that second option, you choose availability over (strong)
consistency.

The second option is not only relevant for multi-datacenter
replication but also more generally for collaboration between
(micro)services where you usually don't want to couple the
availability of one service to that of others. The price for that
is that applications must be able to deal with conflicts, rather
than trying to prevent them. A consistent approach for that is
often missing in distributed applications and often solved with
error-prone ad-hoc solutions. Eventuate tries to change that by
not only
providing APIs to track, detect, and resolve conflicts in an
automated or interactive way but also by providing an
infrastructure

[akka-user] HTTP(S) tunnel for akka-remote

['Martin Krasser' via Akka User List]

Hi,

does anyone have experience or can give recommendations how to set up an 
HTTP(S) tunnel for akka-remote? I'm aware that Netty provides means for 
HTTP tunneling 
. 
Can these be used with akka-remote? Are there other HTTP(S) tunneling 
alternatives you can recommend?


Thanks,
Martin

--
Martin Krasser

blog:http://krasserm.github.io
code:http://github.com/krasserm
twitter: http://twitter.com/mrt1nz

--

 Read the docs: http://akka.io/docs/
 Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
 Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.