Re: message tracker callback in qpid-proton-cpp

[Robbie Gemmell]

I believe the tracker is returned when you call send for a particular
message, and that same tracker is then passed when the
on_tracker_accept etc methods are called after the delivery is
accepted etc. You would use the tracker itself to correlate to a
particular sent message however you wish to.

That package you reference looks like 0.22.0, which is very old now. I
know there are packages of newer releases published at
https://launchpad.net/~qpid/+archive/ubuntu/released, something to
consider.

On Thu, 10 Jun 2021 at 11:09, Xu Wang  wrote:
>
> Hi, proton developers,
>
> I'm using the qpid-proton-cpp library from ubuntu packages[1], and I
> was wondering as for the interface `on_tracker_accept()` inside the
> proton::messaging_handler, how can we learn which message tracker has
> been accepted? I'm planning to integrate some callbacks for sent
> messages.
>
> From the provided examples[1][2], the sender client only records how
> many times `on_tracker_accept()` is called, however, [4] tells that we
> can track a specific message is accepted or not. So how should a
> sender invoke a specific callback for the sent message when dealing
> with different  tracker states?
>
> Please correct me if I'm wrong. And Thanks in advance.
>
> Best,
> Xu
>
> [1] https://packages.ubuntu.com/focal/libqpid-proton-cpp12
> [2] 
> https://github.com/apache/qpid-proton/blob/main/cpp/examples/direct_send.cpp#L75
> [3] 
> https://github.com/apache/qpid-proton/blob/main/cpp/examples/simple_send.cpp#L81
> [4] 
> https://qpid.apache.org/releases/qpid-proton-0.34.0/proton/cpp/api/overview_page.html#autotoc_md8
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

message tracker callback in qpid-proton-cpp

[Xu Wang]

Hi, proton developers,

I'm using the qpid-proton-cpp library from ubuntu packages[1], and I
was wondering as for the interface `on_tracker_accept()` inside the
proton::messaging_handler, how can we learn which message tracker has
been accepted? I'm planning to integrate some callbacks for sent
messages.

>From the provided examples[1][2], the sender client only records how
many times `on_tracker_accept()` is called, however, [4] tells that we
can track a specific message is accepted or not. So how should a
sender invoke a specific callback for the sent message when dealing
with different  tracker states?

Please correct me if I'm wrong. And Thanks in advance.

Best,
Xu

[1] https://packages.ubuntu.com/focal/libqpid-proton-cpp12
[2] 
https://github.com/apache/qpid-proton/blob/main/cpp/examples/direct_send.cpp#L75
[3] 
https://github.com/apache/qpid-proton/blob/main/cpp/examples/simple_send.cpp#L81
[4] 
https://qpid.apache.org/releases/qpid-proton-0.34.0/proton/cpp/api/overview_page.html#autotoc_md8

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Restrict TLS Versions and Ciphers using qpid proton-cpp API

[Justin Ross]

I also took a look, and I don't think we have accessors for these in C++.
I raised a ticket for it:

https://issues.apache.org/jira/browse/PROTON-2139



On Thu, Oct 24, 2019 at 7:21 AM Gordon Sim  wrote:

> On 18/10/2019 3:11 pm, bajanfella wrote:
> > I see that there is a pn_ssl_domain_set_protocol and
> > pn_ssl_domain_set_ciphers in the proton-c API. There there equivalent
> > functionality using the C++ API? How do I get access to the
> pn_ssl_domain_t
> > struct from my CPP API?
>
> If not an expert on the proton c++ API, but I don't see a way to do
> either of these things. Justin, Andrew, am I missing some way of doing so?
>

Re: [Qpid-proton-cpp] Performance regression found in 0.29.0

[Rabih M]

Hi,

Jira issue created: PROTON-2137
<https://issues.apache.org/jira/browse/PROTON-2137>.

Best regards,
Rabih

On Wed, Nov 20, 2019 at 5:01 PM Andrew Stitcher 
wrote:

> Please raise a JIRA with all this information and the reproducer - it
> is very hard to track fixes without this.
>
> Also you could attach your patch, but it is easier in this case to use
> a github PR with a reference to the JIRA. The Apache automation will
> tie them together and this makes it again easier to track.
>
> Andrew
>
> On Tue, 2019-11-19 at 18:04 +, HADI Ali wrote:
> > Hello,
> >
> > After analysis we discovered that the regression is coming from
> > PROTON-2075<
> >
> https://github.com/apache/qpid-proton/commit/e152190459cd75792002d2aae72d351dc22abe27
> >
> > ;: [C++] Allow TLS to use system default trusted certificate.
> > In fact we noticed that the ssl_client_options and the
> > ssl_server_options are not default constructed the same way and that
> > the second one<
> >
> https://github.com/apache/qpid-proton/blob/e152190459cd75792002d2aae72d351dc22abe27/cpp/src/ssl_options.cpp#L99
> > > is calling pni_init_ssl_domain<
> >
> https://github.com/apache/qpid-proton/blob/9dd013335de0694bc52848897b17190f297450c1/c/src/ssl/openssl.c#L475
> > > which is taking some time.
> >
> > What we would like is to avoid initializing ssl when it’s disabled
> > from the connection_options.
> > Does it sound reasonable for you? Should we create a Jira issue and
> > propose a fix?
> >
> > Thanks,
> > Ali & Rabih
> >
> > From: Rabih M 
> > Sent: mercredi 13 novembre 2019 19:22
> > To: users@qpid.apache.org
> > Subject: [Qpid-proton-cpp] Performance regression found in 0.29.0
> >
> > Hello,
> >
> > We are upgrading in our code the proton version from 0.27.0 to
> > 0.29.0.
> > While running our unit tests, we found a considerable performance
> > regression.
> >
> > We were able to reproduce the regression in a very simple use case.
> > Please find the code attached.
> >
> > This test takes 1 ms in the version 0.27.0 and 0.28.0 but it takes 73
> > ms in 0.29.0 .
> >
> > Do you know what might be the cause?
> > We will try to investigate in parallel from our side, too.
> >
> > Thanks,
> > Rabih & Ali
> > ***
> > This e-mail contains information for the intended recipient only. It
> > may contain proprietary material or confidential information. If you
> > are not the intended recipient you are not authorized to distribute,
> > copy or use this e-mail or any attachment to it. Murex cannot
> > guarantee that it is virus free and accepts no responsibility for any
> > loss or damage arising from its use. If you have received this e-mail
> > in error please notify immediately the sender and delete the original
> > email received, any attachments and all copies from your system.
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: [Qpid-proton-cpp] Performance regression found in 0.29.0

[Andrew Stitcher]

Please raise a JIRA with all this information and the reproducer - it
is very hard to track fixes without this.

Also you could attach your patch, but it is easier in this case to use
a github PR with a reference to the JIRA. The Apache automation will
tie them together and this makes it again easier to track.

Andrew

On Tue, 2019-11-19 at 18:04 +, HADI Ali wrote:
> Hello,
> 
> After analysis we discovered that the regression is coming from
> PROTON-2075<
> https://github.com/apache/qpid-proton/commit/e152190459cd75792002d2aae72d351dc22abe27>
> ;: [C++] Allow TLS to use system default trusted certificate.
> In fact we noticed that the ssl_client_options and the
> ssl_server_options are not default constructed the same way and that
> the second one<
> https://github.com/apache/qpid-proton/blob/e152190459cd75792002d2aae72d351dc22abe27/cpp/src/ssl_options.cpp#L99
> > is calling pni_init_ssl_domain<
> https://github.com/apache/qpid-proton/blob/9dd013335de0694bc52848897b17190f297450c1/c/src/ssl/openssl.c#L475
> > which is taking some time.
> 
> What we would like is to avoid initializing ssl when it’s disabled
> from the connection_options.
> Does it sound reasonable for you? Should we create a Jira issue and
> propose a fix?
> 
> Thanks,
> Ali & Rabih
> 
> From: Rabih M 
> Sent: mercredi 13 novembre 2019 19:22
> To: users@qpid.apache.org
> Subject: [Qpid-proton-cpp] Performance regression found in 0.29.0
> 
> Hello,
> 
> We are upgrading in our code the proton version from 0.27.0 to
> 0.29.0.
> While running our unit tests, we found a considerable performance
> regression.
> 
> We were able to reproduce the regression in a very simple use case.
> Please find the code attached.
> 
> This test takes 1 ms in the version 0.27.0 and 0.28.0 but it takes 73
> ms in 0.29.0 .
> 
> Do you know what might be the cause?
> We will try to investigate in parallel from our side, too.
> 
> Thanks,
> Rabih & Ali
> ***
> This e-mail contains information for the intended recipient only. It
> may contain proprietary material or confidential information. If you
> are not the intended recipient you are not authorized to distribute,
> copy or use this e-mail or any attachment to it. Murex cannot
> guarantee that it is virus free and accepts no responsibility for any
> loss or damage arising from its use. If you have received this e-mail 
> in error please notify immediately the sender and delete the original
> email received, any attachments and all copies from your system.


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: [Qpid-proton-cpp] Performance regression found in 0.29.0

[Rabih M]

Hello again,

Here is a patch that we propose. All unit tests are green on the master.

Can you please @Andrew Stitcher check if this patch is aligned with what
you meant to do in (PROTON-2075) ?

Best regards,
Rabih & Ali

On Tue, Nov 19, 2019 at 7:04 PM HADI Ali  wrote:

> Hello,
>
> After analysis we discovered that the regression is coming from
> PROTON-2075<
> https://github.com/apache/qpid-proton/commit/e152190459cd75792002d2aae72d351dc22abe27>:
> [C++] Allow TLS to use system default trusted certificate.
> In fact we noticed that the ssl_client_options and the ssl_server_options
> are not default constructed the same way and that the second one<
> https://github.com/apache/qpid-proton/blob/e152190459cd75792002d2aae72d351dc22abe27/cpp/src/ssl_options.cpp#L99>
> is calling pni_init_ssl_domain<
> https://github.com/apache/qpid-proton/blob/9dd013335de0694bc52848897b17190f297450c1/c/src/ssl/openssl.c#L475>
> which is taking some time.
>
> What we would like is to avoid initializing ssl when it’s disabled from
> the connection_options.
> Does it sound reasonable for you? Should we create a Jira issue and
> propose a fix?
>
> Thanks,
> Ali & Rabih
>
> From: Rabih M 
> Sent: mercredi 13 novembre 2019 19:22
> To: users@qpid.apache.org
> Subject: [Qpid-proton-cpp] Performance regression found in 0.29.0
>
> Hello,
>
> We are upgrading in our code the proton version from 0.27.0 to 0.29.0.
> While running our unit tests, we found a considerable performance
> regression.
>
> We were able to reproduce the regression in a very simple use case.
> Please find the code attached.
>
> This test takes 1 ms in the version 0.27.0 and 0.28.0 but it takes 73 ms
> in 0.29.0 .
>
> Do you know what might be the cause?
> We will try to investigate in parallel from our side, too.
>
> Thanks,
> Rabih & Ali
> ***
> This e-mail contains information for the intended recipient only. It may
> contain proprietary material or confidential information. If you are not
> the intended recipient you are not authorized to distribute, copy or use
> this e-mail or any attachment to it. Murex cannot guarantee that it is
> virus free and accepts no responsibility for any loss or damage arising
> from its use. If you have received this e-mail in error please notify
> immediately the sender and delete the original email received, any
> attachments and all copies from your system.
>
diff --git a/cpp/src/connection_options.cpp b/cpp/src/connection_options.cpp
index 2bf281d1..b61c2c68 100644
--- a/cpp/src/connection_options.cpp
+++ b/cpp/src/connection_options.cpp
@@ -175,7 +175,8 @@ class connection_options::impl {
 }
 } else if (!client && ssl_server_options.set) {
 pn_ssl_t *ssl = pn_ssl(pnt);
-if (pn_ssl_init(ssl, ssl_server_options.value.impl_->pn_domain(), 
NULL)) {
+pn_ssl_domain_t* ssl_domain = ssl_server_options.value.impl_ ? 
ssl_server_options.value.impl_->pn_domain() : NULL;
+if (pn_ssl_init(ssl, ssl_domain, NULL)) {
 throw error(MSG("server SSL/TLS initialization error"));
 }
 }
diff --git a/cpp/src/ssl_options.cpp b/cpp/src/ssl_options.cpp
index bd4d5c17..f74f014e 100644
--- a/cpp/src/ssl_options.cpp
+++ b/cpp/src/ssl_options.cpp
@@ -99,7 +99,7 @@ ssl_server_options::ssl_server_options(
 throw error(MSG("SSL server configuration failure requiring client 
certificates using " << db));
 }
 
-ssl_server_options::ssl_server_options() : impl_(new impl) {}
+ssl_server_options::ssl_server_options() : impl_(0) {}
 
 ssl_client_options::ssl_client_options(const ssl_client_options& x): 
impl_(x.impl_) {
 if (impl_) impl_->incref();

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

RE: [Qpid-proton-cpp] Performance regression found in 0.29.0

[HADI Ali]

Hello,

After analysis we discovered that the regression is coming from 
PROTON-2075<https://github.com/apache/qpid-proton/commit/e152190459cd75792002d2aae72d351dc22abe27>:
 [C++] Allow TLS to use system default trusted certificate.
In fact we noticed that the ssl_client_options and the ssl_server_options are 
not default constructed the same way and that the second 
one<https://github.com/apache/qpid-proton/blob/e152190459cd75792002d2aae72d351dc22abe27/cpp/src/ssl_options.cpp#L99>
 is calling 
pni_init_ssl_domain<https://github.com/apache/qpid-proton/blob/9dd013335de0694bc52848897b17190f297450c1/c/src/ssl/openssl.c#L475>
 which is taking some time.

What we would like is to avoid initializing ssl when it’s disabled from the 
connection_options.
Does it sound reasonable for you? Should we create a Jira issue and propose a 
fix?

Thanks,
Ali & Rabih

From: Rabih M 
Sent: mercredi 13 novembre 2019 19:22
To: users@qpid.apache.org
Subject: [Qpid-proton-cpp] Performance regression found in 0.29.0

Hello,

We are upgrading in our code the proton version from 0.27.0 to 0.29.0.
While running our unit tests, we found a considerable performance regression.

We were able to reproduce the regression in a very simple use case.
Please find the code attached.

This test takes 1 ms in the version 0.27.0 and 0.28.0 but it takes 73 ms in 
0.29.0 .

Do you know what might be the cause?
We will try to investigate in parallel from our side, too.

Thanks,
Rabih & Ali
***
This e-mail contains information for the intended recipient only. It may 
contain proprietary material or confidential information. If you are not the 
intended recipient you are not authorized to distribute, copy or use this 
e-mail or any attachment to it. Murex cannot guarantee that it is virus free 
and accepts no responsibility for any loss or damage arising from its use. If 
you have received this e-mail in error please notify immediately the sender and 
delete the original email received, any attachments and all copies from your 
system.

[Qpid-proton-cpp] Performance regression found in 0.29.0

[Rabih M]

Hello,

We are upgrading in our code the proton version from 0.27.0 to 0.29.0.
While running our unit tests, we found a considerable performance
regression.

We were able to reproduce the regression in a very simple use case.
Please find the code attached.

This test takes 1 ms in the version 0.27.0 and 0.28.0 but it takes 73 ms in
0.29.0 .

Do you know what might be the cause?
We will try to investigate in parallel from our side, too.

Thanks,
Rabih & Ali
#include 
#include 
#include 

#include 

class handler : public proton::messaging_handler {

public:
   handler(const std::string& u) : url(u) {}
private:
   void on_container_start(proton::container& c)override 
   {
  c.connect(url, proton::connection_options().sasl_enabled(false));
   }

   void on_connection_open(proton::connection& c)override 
   {
  c.close();
   }

   std::string url;
};


int main()
{
   try
   {
  handler h("127.0.0.1:1234"); //wrong port
  proton::container(h).run();
   }
   catch (std::exception e)
   {
  std::cout << "Exception thrown at the client side: " << e.what();
   }
   return 0;
}


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Restrict TLS Versions and Ciphers using qpid proton-cpp API

[Gordon Sim]

On 18/10/2019 3:11 pm, bajanfella wrote:

I see that there is a pn_ssl_domain_set_protocol and
pn_ssl_domain_set_ciphers in the proton-c API. There there equivalent
functionality using the C++ API? How do I get access to the pn_ssl_domain_t
struct from my CPP API?


If not an expert on the proton c++ API, but I don't see a way to do 
either of these things. Justin, Andrew, am I missing some way of doing so?



-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Restrict TLS Versions and Ciphers using qpid proton-cpp API

[bajanfella]

I see that there is a pn_ssl_domain_set_protocol and
pn_ssl_domain_set_ciphers in the proton-c API. There there equivalent
functionality using the C++ API? How do I get access to the pn_ssl_domain_t
struct from my CPP API?



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Qpid proton cpp public API

[Andrew Stitcher]

On Wed, 2019-04-03 at 15:19 +0200, Rabih M wrote:
> Hello,
> 
> I am using the latest version of proton cpp 0.27.0.
> 
> I have a suggestion for the signature of the messaging handler's
> methods:
> In my opinion, giving the parameters as references is error prone.
> Because
> it gives the user the illusion that he can keep a reference on the
> proton
> object in his handler and that proton is managing the life cycle of
> this
> object which is not the case. In other words, proton is giving a
> reference
> to a temporary variable.
> 
> I understand that you would like to optimise the copy but now with
> C++ 11
> we can move the objects.
> 
> What do you think?

I think you are mistaken! Passing a reference - merely signals that
there will always be a value here never a null - It doesn't (in my
experience) signal anything about the lifetime of that reference.
Usually things passed into a call are only guaranteed to last for the
duration of the call itself!

In any event you can't keep a reference in any case irrespective of the
passing convention. It seems you think that the following should be
safe:

class Foo {
  proton::connection* connection;

  ...

  void on_connection_blah(connection& c) {
connection = 
  }
}

Huh?

However the following is safe the lifetime of connection will be
extended until the instance of Foo itself is destroyed - if not there
is a bug:

class Foo {
  proton::connection connection;

  ...

  void on_connection_blah(connection& c) {
connection = c;
  }
}

There would be no advantage to making the signatures
  void on_connection_blah(connection)
which would force a copy for every callback. And the signature
  void on_connection_blah(connection&&) is just wrong as we are not
passing the ownership of the connection to the callback.

Andrew



-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-proton-cpp decoder.cpp - possible bug?

[Nathan]

Hi,

Please ignore me - I think this was actually my bug - it only happens if you 
attempt to stream an empty proton::message.

Before it worked - but it looks like it shouldn't have - the new behavior seems 
more correct.

Thanks
Nathan

 
On Mon, Sep 24, 2018 at 9:42 AM Nathan mailto:n...@gmx.es]> wrote:
Hi,

I applied the bug fix below into the code for 0.24.0

It fixes that issue, but appears to have a side effect of causing open ssl to 
disconnect too soon - when the fix below is in place we get "no more data" 
exceptions being thrown from decoder.cpp around line 75, but when the fix below 
is removed the ssl session continues no problem.

This is the throw:

decoder::pre_get() {
if (!next()) throw conversion_error("no more data");

That doesnt happen when the fix below is removed.

Wasnt sure if perhaps I need something else from 0.25 to fix that, or if the 
fix below introduced this new issue?
 
That doesn't ring any bells, possibly this is a regression that we missed in 
testing. Can you open a JIRA with the details of how to reproduce it? I'll get 
on it ASAP.
 
Thanks,
Alan.
 

Thanks
Nathan



From: Alan Conway [mailto:acon...@redhat.com[mailto:acon...@redhat.com]]
Sent: 07 September 2018 16:10
To: 
users@qpid.apache.org[mailto:users@qpid.apache.org][mailto:users@qpid.apache.org[mailto:users@qpid.apache.org]]
Cc: n...@gmx.es[mailto:n...@gmx.es][mailto:n...@gmx.es[mailto:n...@gmx.es]]
Subject: Re: qpid-proton-cpp decoder.cpp - possible bug?

I think this is a bug I fixed recently, it should be in the latest release 
proton-0.25. If you still have a problem please raise a JIRA.

Here's the fix - NULL is somewhat special but it is a valid scalar type, so I 
added it to type_id_is_scalar()

https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176[https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176][https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176[https://github.com/alanconway/qpid-proton/commit/9e8edc17%23diff-d6a2b218a8187976430ae388c2a9b176]]



On Thu, Sep 6, 2018 at 7:55 AM, 
mailto:n...@gmx.es][mailto:n...@gmx.es[mailto:n...@gmx.es]]> wrote:
Hi,

Around line 180, decoder.cpp has the following:


decoder& decoder::operator>>(scalar& x) {
internal::state_guard sg(*this);
type_id got = pre_get();

if (!type_id_is_scalar(got))
throw conversion_error("expected scalar, found "+type_name(got));

x.set(pn_data_get_atom(pn_object()));
sg.cancel(); // No error, no rewind
return *this;
}


When our client code is talking to a 3rd party system's broker, we find that 
the scalar in argument "x" is coming through from the other party as type NULL 
in some cases - which causes the throw in the above method to get triggered and 
then disconnects us from the broker.

However, if we comment out the if/throw from the code in decoder.cpp, 
everything then appears to be workding 100% correctly.

Is there a correct way to do this without removing the throw? - it seems that 
if the check is there it must have a purpose, so by removing the check I am 
probably opening myself up to some other issues further down the line?

Thanks
N


-
To unsubscribe, e-mail: 
users-unsubscr...@qpid.apache.org[mailto:users-unsubscr...@qpid.apache.org]
For additional commands, e-mail: 
users-h...@qpid.apache.org[mailto:users-h...@qpid.apache.org]

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-proton-cpp decoder.cpp - possible bug?

[Alan Conway]

On Mon, Sep 24, 2018 at 9:42 AM Nathan  wrote:

> Hi,
>
> I applied the bug fix below into the code for 0.24.0
>
> It fixes that issue, but appears to have a side effect of causing open ssl
> to disconnect too soon - when the fix below is in place we get "no more
> data" exceptions being thrown from decoder.cpp around line 75, but when the
> fix below is removed the ssl session continues no problem.
>
> This is the throw:
>
> decoder::pre_get() {
> if (!next()) throw conversion_error("no more data");
>
> That doesnt happen when the fix below is removed.
>
> Wasnt sure if perhaps I need something else from 0.25 to fix that, or if
> the fix below introduced this new issue?
>

That doesn't ring any bells, possibly this is a regression that we missed
in testing. Can you open a JIRA with the details of how to reproduce it?
I'll get on it ASAP.

Thanks,
Alan.


> Thanks
> Nathan
>
>
>
> From: Alan Conway [mailto:acon...@redhat.com]
> Sent: 07 September 2018 16:10
> To: users@qpid.apache.org[mailto:users@qpid.apache.org]
> Cc: n...@gmx.es[mailto:n...@gmx.es]
> Subject: Re: qpid-proton-cpp decoder.cpp - possible bug?
>
> I think this is a bug I fixed recently, it should be in the latest release
> proton-0.25. If you still have a problem please raise a JIRA.
>
> Here's the fix - NULL is somewhat special but it is a valid scalar type,
> so I added it to type_id_is_scalar()
>
>
> https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176[https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176]
>
>
>
> On Thu, Sep 6, 2018 at 7:55 AM, mailto:n...@gmx.es]> wrote:
> Hi,
>
> Around line 180, decoder.cpp has the following:
>
>
> decoder& decoder::operator>>(scalar& x) {
> internal::state_guard sg(*this);
> type_id got = pre_get();
>
> if (!type_id_is_scalar(got))
> throw conversion_error("expected scalar, found "+type_name(got));
>
> x.set(pn_data_get_atom(pn_object()));
> sg.cancel(); // No error, no rewind
> return *this;
> }
>
>
> When our client code is talking to a 3rd party system's broker, we find
> that the scalar in argument "x" is coming through from the other party as
> type NULL in some cases - which causes the throw in the above method to get
> triggered and then disconnects us from the broker.
>
> However, if we comment out the if/throw from the code in decoder.cpp,
> everything then appears to be workding 100% correctly.
>
> Is there a correct way to do this without removing the throw? - it seems
> that if the check is there it must have a purpose, so by removing the check
> I am probably opening myself up to some other issues further down the line?
>
> Thanks
> N
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: qpid-proton-cpp decoder.cpp - possible bug?

[Nathan]

Hi,
 
I applied the bug fix below into the code for 0.24.0
 
It fixes that issue, but appears to have a side effect of causing open ssl to 
disconnect too soon - when the fix below is in place we get "no more data" 
exceptions being thrown from decoder.cpp around line 75, but when the fix below 
is removed the ssl session continues no problem.
 
This is the throw:
 
decoder::pre_get() {
if (!next()) throw conversion_error("no more data");
 
That doesnt happen when the fix below is removed.
 
Wasnt sure if perhaps I need something else from 0.25 to fix that, or if the 
fix below introduced this new issue?
 
Thanks
Nathan


 
From: Alan Conway [mailto:acon...@redhat.com]
Sent: 07 September 2018 16:10
To: users@qpid.apache.org[mailto:users@qpid.apache.org]
Cc: n...@gmx.es[mailto:n...@gmx.es]
Subject: Re: qpid-proton-cpp decoder.cpp - possible bug?
 
I think this is a bug I fixed recently, it should be in the latest release 
proton-0.25. If you still have a problem please raise a JIRA.
 
Here's the fix - NULL is somewhat special but it is a valid scalar type, so I 
added it to type_id_is_scalar()
 
https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176[https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176]
 
 
 
On Thu, Sep 6, 2018 at 7:55 AM, mailto:n...@gmx.es]> wrote:
Hi,
 
Around line 180, decoder.cpp has the following:
 
 
decoder& decoder::operator>>(scalar& x) {
internal::state_guard sg(*this);
type_id got = pre_get();
 
if (!type_id_is_scalar(got))
throw conversion_error("expected scalar, found "+type_name(got));
 
x.set(pn_data_get_atom(pn_object()));
sg.cancel(); // No error, no rewind
return *this;
}
 
 
When our client code is talking to a 3rd party system's broker, we find that 
the scalar in argument "x" is coming through from the other party as type NULL 
in some cases - which causes the throw in the above method to get triggered and 
then disconnects us from the broker.
 
However, if we comment out the if/throw from the code in decoder.cpp, 
everything then appears to be workding 100% correctly.
 
Is there a correct way to do this without removing the throw? - it seems that 
if the check is there it must have a purpose, so by removing the check I am 
probably opening myself up to some other issues further down the line?
 
Thanks
N
 

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-proton-cpp decoder.cpp - possible bug?

[Nathan]

This is great - many thanks for the help!
 

From: Alan Conway [mailto:acon...@redhat.com] 
Sent: 07 September 2018 16:10
To: users@qpid.apache.org
Cc: n...@gmx.es
Subject: Re: qpid-proton-cpp decoder.cpp - possible bug?

I think this is a bug I fixed recently, it should be in the latest release 
proton-0.25. If you still have a problem please raise a JIRA.

Here's the fix - NULL is somewhat special but it is a valid scalar type, so I 
added it to type_id_is_scalar()

https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176



On Thu, Sep 6, 2018 at 7:55 AM,  wrote:
Hi,

Around line 180, decoder.cpp has the following:


decoder& decoder::operator>>(scalar& x) {
internal::state_guard sg(*this);
type_id got = pre_get();
  
if (!type_id_is_scalar(got))
throw conversion_error("expected scalar, found "+type_name(got));

x.set(pn_data_get_atom(pn_object()));
sg.cancel();// No error, no rewind
return *this;
}


When our client code is talking to a 3rd party system's broker, we find that 
the scalar in argument "x" is coming through from the other party as type NULL 
in some cases - which causes the throw in the above method to get triggered and 
then disconnects us from the broker.

However, if we comment out the if/throw from the code in decoder.cpp, 
everything then appears to be workding 100% correctly.

Is there a correct way to do this without removing the throw? - it seems that 
if the check is there it must have a purpose, so by removing the check I am 
probably opening myself up to some other issues further down the line?

Thanks
N

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-proton-cpp decoder.cpp - possible bug?

[Alan Conway]

I think this is a bug I fixed recently, it should be in the latest release
proton-0.25. If you still have a problem please raise a JIRA.

Here's the fix - NULL is somewhat special but it is a valid scalar type, so
I added it to type_id_is_scalar()

https://github.com/alanconway/qpid-proton/commit/9e8edc17#diff-d6a2b218a8187976430ae388c2a9b176



On Thu, Sep 6, 2018 at 7:55 AM,  wrote:

> Hi,
>
> Around line 180, decoder.cpp has the following:
>
>
> decoder& decoder::operator>>(scalar& x) {
> internal::state_guard sg(*this);
> type_id got = pre_get();
>
> if (!type_id_is_scalar(got))
> throw conversion_error("expected scalar, found "+type_name(got));
>
> x.set(pn_data_get_atom(pn_object()));
> sg.cancel();// No error, no rewind
> return *this;
> }
>
>
> When our client code is talking to a 3rd party system's broker, we find
> that the scalar in argument "x" is coming through from the other party as
> type NULL in some cases - which causes the throw in the above method to get
> triggered and then disconnects us from the broker.
>
> However, if we comment out the if/throw from the code in decoder.cpp,
> everything then appears to be workding 100% correctly.
>
> Is there a correct way to do this without removing the throw? - it seems
> that if the check is there it must have a purpose, so by removing the check
> I am probably opening myself up to some other issues further down the line?
>
> Thanks
> N
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

qpid-proton-cpp decoder.cpp - possible bug?

[nmk]

Hi,

Around line 180, decoder.cpp has the following:


decoder& decoder::operator>>(scalar& x) {
    internal::state_guard sg(*this);
    type_id got = pre_get();
  
    if (!type_id_is_scalar(got))
        throw conversion_error("expected scalar, found "+type_name(got));

    x.set(pn_data_get_atom(pn_object()));
    sg.cancel();                // No error, no rewind
    return *this;
}


When our client code is talking to a 3rd party system's broker, we find that 
the scalar in argument "x" is coming through from the other party as type NULL 
in some cases - which causes the throw in the above method to get triggered and 
then disconnects us from the broker.

However, if we comment out the if/throw from the code in decoder.cpp, 
everything then appears to be workding 100% correctly.

Is there a correct way to do this without removing the throw? - it seems that 
if the check is there it must have a purpose, so by removing the check I am 
probably opening myself up to some other issues further down the line?

Thanks
N

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-proton cpp windows : Crash

[Baptiste Robert]

Update:

I opened a JIRA : https://issues.apache.org/jira/browse/PROTON-1833


2018-04-23 16:50 GMT+02:00 Alan Conway :

> On Mon, Apr 23, 2018 at 5:37 AM, Baptiste Robert <
> baptisterober...@gmail.com
> > wrote:
>
> > Hello,
> >
> > I'm encountering a crash in the proton library. What I'm doing ? Just
> > creating a proton::container, listening locally on 0.0.0.0:5672 and then
> > call stop() on the container and getting outside of the scope (object is
> > then destroy) => the crash happen.
> >
> > Where does it crash ? In *win_iocp.c*, the line in red
> >
> > void pn_proactor_free(pn_proactor_t *p) {
> >   *DeleteTimerQueueEx(p->timer_queue, INVALID_HANDLE_VALUE);*
> >   DeleteCriticalSection(>timer_lock);
> >   DeleteCriticalSection(>bind_lock);
> >   proactor_shutdown(p);
> >
> >   delete p->reaper;
> >   WSACleanup();
> >   pn_collector_free(p->collector);
> >   free(p);
> > }
> >
> >
> > Proton version : 0.21
> > Windows 7 - 64 bits
> > Visual studio 2010
> >
> > Does any one have a clue ? It remind me this issue on the dispatch
> router:
> > https://issues.apache.org/jira/browse/DISPATCH-945
> >
> >
> Sounds like a bug. Can you raise a JIRA and attach your code to reproduce?
> The dispatch issue is not quite the same (dispatch has 2 server loops, one
> for AMQP and one for HTTP) but I believe we did fix a similar issue in the
> epoll proactor a while back.
>
> --
>
> > Baptiste Robert
> >
>



-- 
Baptiste Robert

Re: qpid-proton cpp windows : Crash

[Alan Conway]

On Mon, Apr 23, 2018 at 5:37 AM, Baptiste Robert  wrote:

> Hello,
>
> I'm encountering a crash in the proton library. What I'm doing ? Just
> creating a proton::container, listening locally on 0.0.0.0:5672 and then
> call stop() on the container and getting outside of the scope (object is
> then destroy) => the crash happen.
>
> Where does it crash ? In *win_iocp.c*, the line in red
>
> void pn_proactor_free(pn_proactor_t *p) {
>   *DeleteTimerQueueEx(p->timer_queue, INVALID_HANDLE_VALUE);*
>   DeleteCriticalSection(>timer_lock);
>   DeleteCriticalSection(>bind_lock);
>   proactor_shutdown(p);
>
>   delete p->reaper;
>   WSACleanup();
>   pn_collector_free(p->collector);
>   free(p);
> }
>
>
> Proton version : 0.21
> Windows 7 - 64 bits
> Visual studio 2010
>
> Does any one have a clue ? It remind me this issue on the dispatch router:
> https://issues.apache.org/jira/browse/DISPATCH-945
>
>
Sounds like a bug. Can you raise a JIRA and attach your code to reproduce?
The dispatch issue is not quite the same (dispatch has 2 server loops, one
for AMQP and one for HTTP) but I believe we did fix a similar issue in the
epoll proactor a while back.

-- 

> Baptiste Robert
>

qpid-proton cpp windows : Crash

[Baptiste Robert]

Hello,

I'm encountering a crash in the proton library. What I'm doing ? Just
creating a proton::container, listening locally on 0.0.0.0:5672 and then
call stop() on the container and getting outside of the scope (object is
then destroy) => the crash happen.

Where does it crash ? In *win_iocp.c*, the line in red

void pn_proactor_free(pn_proactor_t *p) {
  *DeleteTimerQueueEx(p->timer_queue, INVALID_HANDLE_VALUE);*
  DeleteCriticalSection(>timer_lock);
  DeleteCriticalSection(>bind_lock);
  proactor_shutdown(p);

  delete p->reaper;
  WSACleanup();
  pn_collector_free(p->collector);
  free(p);
}


Proton version : 0.21
Windows 7 - 64 bits
Visual studio 2010

Does any one have a clue ? It remind me this issue on the dispatch router:
https://issues.apache.org/jira/browse/DISPATCH-945

-- 
Baptiste Robert

Re: QPid-proton cpp 0.21 - Crash

[Baptiste Robert]

I do not use any kind of timer on my side, but maybe it's a similar
internal proton issue as this one :
https://issues.apache.org/jira/browse/DISPATCH-945



2018-04-18 2:21 GMT+02:00 Chris Richardson :

> A couple of suggestions spring to mind - I've experienced problems with
> timers in other libraries where a timer fires after (or indeed during) its
> callback or associated data has been deleted, resulting in a segfault.
> Could this be relevant? Probably capturing a core dump and inspecting with
> gdb would be enlightening and would be my first port of call. Another
> approach might be some code introspection to verify that timers are
> cancelled and that their handlers have completed before relevant garbage
> collection takes place. Rather general comments I'm afraid but I thought it
> might be worth consideration.
>
> Chris
>
>
>
> On 17 April 2018 at 16:36, Baptiste Robert 
> wrote:
>
> > Hello,
> >
> > When I create a proton::container and use it, I have a crash when I
> delete
> > the proton object:
> >
> > void pn_proactor_free(pn_proactor_t *p) {
> > ->  DeleteTimerQueueEx(p->timer_queue, INVALID_HANDLE_VALUE);
> >
> > I'm using proton 0.21 compiled in CXX03 mode.
> >
> > Is anyone have an idea ?
> >
> > Thank you,
> >
> > Baptiste
> >
>



-- 
Baptiste Robert

Re: QPid-proton cpp 0.21 - Crash

[Chris Richardson]

A couple of suggestions spring to mind - I've experienced problems with
timers in other libraries where a timer fires after (or indeed during) its
callback or associated data has been deleted, resulting in a segfault.
Could this be relevant? Probably capturing a core dump and inspecting with
gdb would be enlightening and would be my first port of call. Another
approach might be some code introspection to verify that timers are
cancelled and that their handlers have completed before relevant garbage
collection takes place. Rather general comments I'm afraid but I thought it
might be worth consideration.

Chris



On 17 April 2018 at 16:36, Baptiste Robert 
wrote:

> Hello,
>
> When I create a proton::container and use it, I have a crash when I delete
> the proton object:
>
> void pn_proactor_free(pn_proactor_t *p) {
> ->  DeleteTimerQueueEx(p->timer_queue, INVALID_HANDLE_VALUE);
>
> I'm using proton 0.21 compiled in CXX03 mode.
>
> Is anyone have an idea ?
>
> Thank you,
>
> Baptiste
>

QPid-proton cpp 0.21 - Crash

[Baptiste Robert]

Hello,

When I create a proton::container and use it, I have a crash when I delete
the proton object:

void pn_proactor_free(pn_proactor_t *p) {
->  DeleteTimerQueueEx(p->timer_queue, INVALID_HANDLE_VALUE);

I'm using proton 0.21 compiled in CXX03 mode.

Is anyone have an idea ?

Thank you,

Baptiste

Re: qpid-proton cpp - Wrong auth with reconnect options lead to error : Too many reconnect attempts

[Justin Ross]

Hi, Baptiste.  There's something coming soon in Proton C 0.22.0 that may
help with that.

https://issues.apache.org/jira/browse/PROTON-1589

This is on master now, and the 0.22.0 release process will start this month.

On Fri, Mar 23, 2018 at 7:57 AM, Baptiste Robert  wrote:

> Hello,
>
> I've an annoying issue when I define reconnect_options and call connect
> method on the proton::container with this option. If the authentication is
> wrong (bad password/login), I got the following error : [proton:io] - [Too
> many reconnect attempts (40)]
> In my use case, I really want to know if it fails because the auth was bad
> or if the server is just unreachable.
> If I remove the reconnect_options then I get this information, but why I
> cannot get both ?
>
> Thank you,
>
> --
> Baptiste Robert
>

qpid-proton cpp - Wrong auth with reconnect options lead to error : Too many reconnect attempts

[Baptiste Robert]

Hello,

I've an annoying issue when I define reconnect_options and call connect
method on the proton::container with this option. If the authentication is
wrong (bad password/login), I got the following error : [proton:io] - [Too
many reconnect attempts (40)]
In my use case, I really want to know if it fails because the auth was bad
or if the server is just unreachable.
If I remove the reconnect_options then I get this information, but why I
cannot get both ?

Thank you,

-- 
Baptiste Robert

Re: qpid-proton cpp - Broker example segfault

[Alan Conway]

On Mon, Mar 19, 2018 at 12:07 PM, Baptiste Robert <
baptisterober...@gmail.com> wrote:

> Thank you for your answer, It fixed my issues. Maybe it's due tthe
> partial support of C++11 integrated in GCC 4.6.2.
>
In the same way, is it possible to build the last qpid-proton version with
> OpenSSL 1.0.0 ? Since the 0.19 version I cannot built it, is it possible to
> set a cmake variable to make it compliant ?
>
>
I'm glad you got the C++ issue sorted out. I don't know about the SSL
problem. It is probably possible to work around it but I'm not aware of a
fix already in the cmake files.Perhaps someone else on the list can answer
that.



> Thank you,
>
>
> 2018-03-19 15:21 GMT+01:00 Alan Conway :
>
> > On Mon, Mar 19, 2018 at 4:45 AM, Baptiste Robert <
> > baptisterober...@gmail.com
> > > wrote:
> >
> > > Hello,
> > >
> > > I'm having an issue with the broker example. When I run it works, but a
> > > soon as I send messages to it I get a segfault. I send messages with
> the
> > > examples simple_send.
> > >
> > > Note : I compiled the broker with gcc 4.6.2 e.g. without C++11 support,
> > and
> > > I'm supposing that is the root cause of a lot of my problem with qpid..
> > >
> >
> > Before C++11 you can't use multiple threads, but the broker and other
> > examples are supposed to work single-threaded in that case so we should
> > find out what's going wrong.
> >
> > Try a new build with `cmake -DBUILD_CPP_03=YES` to tell cmake that you
> want
> > to build in c++-03 compatibility mode. Cmake is supposed to figure that
> out
> > for itself but maybe there's a problem in the compiler detection logic.
> >
> > If it still crashes raise a JIRA at issues.apache.org and attache
> > theCMakeCache.txt file generated by cmake and the output of these
> commands:
> >
> > $ gcc -v
> > $ gdb --batch --quiet -ex "thread apply all bt full" -ex "quit"
> >  
> >
> >
> > > --
> > > Baptiste Robert
> > >
> >
>
>
>
> --
> Baptiste Robert
>

Re: qpid-proton cpp - Broker example segfault

[Baptiste Robert]

Thank you for your answer, It fixed my issues. Maybe it's due to the
partial support of C++11 integrated in GCC 4.6.2.
In the same way, is it possible to build the last qpid-proton version with
OpenSSL 1.0.0 ? Since the 0.19 version I cannot built it, is it possible to
set a cmake variable to make it compliant ?

Thank you,


2018-03-19 15:21 GMT+01:00 Alan Conway :

> On Mon, Mar 19, 2018 at 4:45 AM, Baptiste Robert <
> baptisterober...@gmail.com
> > wrote:
>
> > Hello,
> >
> > I'm having an issue with the broker example. When I run it works, but a
> > soon as I send messages to it I get a segfault. I send messages with the
> > examples simple_send.
> >
> > Note : I compiled the broker with gcc 4.6.2 e.g. without C++11 support,
> and
> > I'm supposing that is the root cause of a lot of my problem with qpid..
> >
>
> Before C++11 you can't use multiple threads, but the broker and other
> examples are supposed to work single-threaded in that case so we should
> find out what's going wrong.
>
> Try a new build with `cmake -DBUILD_CPP_03=YES` to tell cmake that you want
> to build in c++-03 compatibility mode. Cmake is supposed to figure that out
> for itself but maybe there's a problem in the compiler detection logic.
>
> If it still crashes raise a JIRA at issues.apache.org and attache
> theCMakeCache.txt file generated by cmake and the output of these commands:
>
> $ gcc -v
> $ gdb --batch --quiet -ex "thread apply all bt full" -ex "quit"
>  
>
>
> > --
> > Baptiste Robert
> >
>



-- 
Baptiste Robert

Re: qpid-proton cpp - Broker example segfault

[Alan Conway]

On Mon, Mar 19, 2018 at 4:45 AM, Baptiste Robert  wrote:

> Hello,
>
> I'm having an issue with the broker example. When I run it works, but a
> soon as I send messages to it I get a segfault. I send messages with the
> examples simple_send.
>
> Note : I compiled the broker with gcc 4.6.2 e.g. without C++11 support, and
> I'm supposing that is the root cause of a lot of my problem with qpid..
>

Before C++11 you can't use multiple threads, but the broker and other
examples are supposed to work single-threaded in that case so we should
find out what's going wrong.

Try a new build with `cmake -DBUILD_CPP_03=YES` to tell cmake that you want
to build in c++-03 compatibility mode. Cmake is supposed to figure that out
for itself but maybe there's a problem in the compiler detection logic.

If it still crashes raise a JIRA at issues.apache.org and attache
theCMakeCache.txt file generated by cmake and the output of these commands:

$ gcc -v
$ gdb --batch --quiet -ex "thread apply all bt full" -ex "quit"
 


> --
> Baptiste Robert
>

qpid-proton cpp - Broker example segfault

[Baptiste Robert]

Hello,

I'm having an issue with the broker example. When I run it works, but a
soon as I send messages to it I get a segfault. I send messages with the
examples simple_send.

Note : I compiled the broker with gcc 4.6.2 e.g. without C++11 support, and
I'm supposing that is the root cause of a lot of my problem with qpid..

Thank you,

-- 
Baptiste Robert

Re: qpid proton-cpp windows

[Keith W]

On 15 March 2018 at 13:42, Alan Conway  wrote:
>
>
> On Thu, Mar 15, 2018 at 9:28 AM, Baptiste Robert
>  wrote:
>>
>> Hello everybody,
>>
>> I currently have an issue with the qpid-proton c++ implementation on
>> windows. I can't connect to a Java broker with an authenticationproviders
>> set to : PlainPasswordFile.
>>
>> I built the simple_send examples and I can send messages to the broker if
>> authenticationproviders is set to Anonymous. As soon as I reactivate
>> PlainPasswordFile then I do not have any exception and exit code is set to
>> 1, no messages are sent, and Y is display in the console.
>
>
> If you are using the SASL plain mechanism without TLS/SSL you also need to
> set the connection_option sasl_allow_insecure_mechs(true)

Likewise, Broker-J won't offer the PLAIN SASL mechanism over a non-TLS
port to a peer.   If you don't want to use TLS, it can be overridden
using instructions that talk about overriding secureOnlyMechanisms
you'll find here:

https://qpid.apache.org/releases/qpid-broker-j-7.0.2/book/Java-Broker-Security.html#Java-Broker-Security-Authentication-Providers

>
> By default, SASL will disallow mechanisms that send plain-text passwords
> unless the connection is encrypted - even though you will see PLAIN listed
> in the set of allowed mechanisms from the server, it will ignored by the
> client.
>
> See the attached example which lets you play with SASL settings. Also a tip:
> set PN_TRACE_FRM=1 in your environment to get more information about the
> connection negotiation, including  some SASL info.
>
>>
>> Maybe I miss a dependency ?
>>
>> Baptiste
>
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid proton-cpp windows

[Alan Conway]

On Thu, Mar 15, 2018 at 9:28 AM, Baptiste Robert  wrote:

> Hello everybody,
>
> I currently have an issue with the qpid-proton c++ implementation on
> windows. I can't connect to a Java broker with an authenticationproviders
> set to : PlainPasswordFile.
>
> I built the simple_send examples and I can send messages to the broker if
> authenticationproviders is set to Anonymous. As soon as I reactivate
> PlainPasswordFile then I do not have any exception and exit code is set to
> 1, no messages are sent, and Y is display in the console.
>

If you are using the SASL plain mechanism without TLS/SSL you also need to
set the connection_option sasl_allow_insecure_mechs(true)

By default, SASL will disallow mechanisms that send plain-text passwords
unless the connection is encrypted - even though you will see PLAIN listed
in the set of allowed mechanisms from the server, it will ignored by the
client.

See the attached example which lets you play with SASL settings. Also a
tip: set PN_TRACE_FRM=1 in your environment to get more information about
the connection negotiation, including  some SASL info.


> Maybe I miss a dependency ?
>
> Baptiste
>
/*
 *
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *   http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing,
 * software distributed under the License is distributed on an
 * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 * KIND, either express or implied.  See the License for the
 * specific language governing permissions and limitations
 * under the License.
 *
 */

#include "options.hpp"

#include 
#include 
#include 
#include 

#include 

#include "fake_cpp11.hpp"

class simple_connect : public proton::messaging_handler {
  private:
std::string url;
std::string user;
std::string password;
bool sasl;
std::string mechs;
bool insecure;
proton::connection connection;

  public:
simple_connect(const std::string , const std::string , const std::string , bool s, const std::string& ms, bool in) :
url(a), user(u), password(p), sasl(s), mechs(ms), insecure(in) {}

void on_container_start(proton::container ) OVERRIDE {
proton::connection_options co;
if (!user.empty()) co.user(user);
if (!password.empty()) co.password(password);
if (sasl) co.sasl_enabled(true);
if (!mechs.empty()) co.sasl_allowed_mechs(mechs);
co.sasl_allow_insecure_mechs(insecure);
connection = c.connect(url, co);
}

void on_connection_open(proton::connection ) OVERRIDE {
c.close();
}

void on_error(const proton::error_condition& e) OVERRIDE {
throw std::runtime_error(e.what());
}
};

int main(int argc, char **argv) {
std::string address("127.0.0.1:5672/examples");
std::string user;
std::string password;
std::string mechs;
bool sasl = false;
bool insecure = false;
example::options opts(argc, argv);

opts.add_value(address, 'a', "address", "connect and send to URL", "URL");
opts.add_value(user, 'u', "user", "authenticate as USER", "USER");
opts.add_value(password, 'p', "password", "authenticate with PASSWORD", "PASSWORD");
opts.add_flag(sasl,'s', "sasl", "force SASL authentication with no user specified (Use for Kerberos/GSSAPI)");
opts.add_value(mechs, 'm', "mechs", "allowed SASL mechanisms", "MECHS");
opts.add_flag(insecure, 'i', "insecure", "allow clear-text passwords");

try {
opts.parse();

simple_connect connect(address, user, password, sasl, mechs, insecure);
proton::container(connect).run();

return 0;
} catch (const example::bad_option& e) {
std::cout << opts << std::endl << e.what() << std::endl;
} catch (const std::exception& e) {
std::cerr << e.what() << std::endl;
}

return 1;
}

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

qpid proton-cpp windows

[Baptiste Robert]

Hello everybody,

I currently have an issue with the qpid-proton c++ implementation on
windows. I can't connect to a Java broker with an authenticationproviders
set to : PlainPasswordFile.

I built the simple_send examples and I can send messages to the broker if
authenticationproviders is set to Anonymous. As soon as I reactivate
PlainPasswordFile then I do not have any exception and exit code is set to
1, no messages are sent, and Y is display in the console.

Maybe I miss a dependency ?

Baptiste

Re: QPid Proton CPP

[Alan Conway]

On Fri, Feb 23, 2018 at 5:43 AM, Baptiste Robert  wrote:

> Hi everybody,
>
> I'm struggling a little bit with the global architecture of proton cpp. I'm
> trying to code a wrapper to simplify the utilization for a end user.
> This wrapper is divided in two classes, a "client" that launch the
> proton::container in a separate thread, and a handler that inherate
> proton::messaging_handler. My "client" part is supposed to allow the user
> to create a new connection, send message, close the connection, etc..
> But I encounter strange behavior from proton, like why when I called a
> close on a connection, then my container exit from his thread (return from
> the run) ?
>

Do  you have some sample code? Possibly there is some confusion about the
threading rules. For example to close a connection from a thread other than
the event handling thread, you need to use a work-queue.
Have you looked at the cpp/multithreaded_* examples?



Same issue with the proton::reconnection_options, it does not works
> properly, if I simulate a broker crash and then relaunch it, my connection
> wait while the container is offline but after the restart I get a transport
> error.
>
It should work - if you can show some sample code we can figure out why
not.

>
> So I supposed my problem here is that I not fully understand the philosophy
> of proton. In my mind it was : one container that handle multiple
> connections, and the container can close/open connections from different
> thread.
>
That's correct, although there are some rules about what you can do from
what thread. Take a look at
proton-c/bindings/cpp/docs/mt.md


>
> Oh and lastly, I do not understand why the proton::error_condition does not
> have an error code (proper enum, not string).
>

It represents an AMQP error condition which has a name, description and
some optional "info" - not an operating system error. In cases where we
generate error conditions internally because of OS errors we usually
include the OS error description in the description but there's no access
to an OS-specific error code.

The condition name acts a bit like an error "type", but it is not a closed
set. For IO related errors we use "proton:io", there are some AMQP standard
names which can be used for errors on the wire, but an application is free
to invent others:

InternalError = "amqp:internal-error"
NotFound  = "amqp:not-found"
UnauthorizedAccess= "amqp:unauthorized-access"
DecodeError   = "amqp:decode-error"
ResourceLimitExceeded = "amqp:resource-limit-exceeded"
NotAllowed= "amqp:not-allowed"
InvalidField  = "amqp:invalid-field"
NotImplemented= "amqp:not-implemented"
ResourceLocked= "amqp:resource-locked"
PreconditionFailed= "amqp:precondition-failed"
ResourceDeleted   = "amqp:resource-deleted"
IllegalState  = "amqp:illegal-state"
FrameSizeTooSmall = "amqp:frame-size-too-small"

QPid Proton CPP

[Baptiste Robert]

Hi everybody,

I'm struggling a little bit with the global architecture of proton cpp. I'm
trying to code a wrapper to simplify the utilization for a end user.
This wrapper is divided in two classes, a "client" that launch the
proton::container in a separate thread, and a handler that inherate
proton::messaging_handler. My "client" part is supposed to allow the user
to create a new connection, send message, close the connection, etc..
But I encounter strange behavior from proton, like why when I called a
close on a connection, then my container exit from his thread (return from
the run) ?
Same issue with the proton::reconnection_options, it does not works
properly, if I simulate a broker crash and then relaunch it, my connection
wait while the container is offline but after the restart I get a transport
error.

So I supposed my problem here is that I not fully understand the philosophy
of proton. In my mind it was : one container that handle multiple
connections, and the container can close/open connections from different
thread.

Oh and lastly, I do not understand why the proton::error_condition does not
have an error code (proper enum, not string).

Thanks you,

-- 
Baptiste Robert

Re: Qpid proton cpp

[Alan Conway]

Possible you've found a bug, but also possible you're hitting some C++
template type-mismatch potholes. C++ compilers often give incomprehensible
error messages for such problems. Post a code snip that illustrates the
problem with the compiler version you're using, and the error message you
see.

If you can choose your compiler for development I recommend clang++ for
better than average error messages. My employer ships software built with
gcc, but I sometimes switch to clang just to figure out what gcc is
complaining about.

On Wed, Feb 7, 2018 at 3:51 AM, Baptiste Robert <baptisterober...@gmail.com>
wrote:

> Hi everybody,
>
> I'm currently working with the qpid proton cpp API (0.18 version) with an
> old gcc without c++11 support. I can compile it and use the examples, but
> I'm building a multi-threaded client, where I'm trying to create new sender
> or sending message asynchronously. So in the examples I found the
> working_queue principles, but I'm facing the following difficulty :
> How to create a work for the work queue properly ?
>
> I found this function proton::make_work, but when I'm trying to call it for
> sending a message like that : proton::make_work(::sender::send,
> _sender, m) it doesn't compile..
>
> I'm missing something, on the asynchronous utilization ?
>
> Have a good day,
>
> --
> Baptiste Robert
>

Re: [qpid-proton-cpp] default_container does not implement thread safety at all?

[Alan Conway]

On Fri, 2017-05-26 at 03:06 +0300, Fj wrote:
> By the way, if anyone is interested, as a stop-gap solution I ended
> up just
> implementing a timer-based callback reenqueuing itself every 50ms and
> pulling callbacks from a properly synchronized queue and executing
> them on
> the worker thread. It's not even that bad performance-wise, it just
> adds
> those 50/2 ms of latency on average.

That's the right workaround. The new C++ API has thread-safe work-
queues that will allow you to have a function executed in the proper
context which will be more efficient and more flexible than using the
timer callbacks. Behind the scenes it uses whatever is efficient on the
underlying IO platform (linux eventfds, libuv async_send etc.) to do
the wakeup.
 
> 
> A better solution would be to do the same thing the Python binding
> does,
> expose a Selectable interface and use it to wake up the reactor
> thread.
> 
> And by the way there's a thing that you might want to consider:
> there's a
> bunch of use cases that don't need inter-thread synchronization, for
> example, single-threaded workers, or multi-threaded workers that only
> need
> to know when to stop and they know it from the socket being closed.
> In
> those cases exposing a thread-safe Container.inject(callback) is a
> total
> waste, because it means that everything else must constantly take
> locks for
> no reason.

+1, those are all good observations. Look at the C proactors to see how
we are implementing this. Suggestions welcome. At the C level all you
can do is trigger a wakeup, to keep the C layer easy to re-implement.
The C++ binding will use the basic wakeup provided by C, and add
programmer convenience in the form of work queues so you can not only
trigger a wakeup but provide a function to be executed.

> 
> Maybe in the spirit of "you don't pay for what you don't use" the way
> Python binding does it is actually fundamentally better, with an
> EventInjector thingie that you can add to your Container if you want
> and
> then call injector.inject(callback) instead of container.inject?

That's roughly how it works: there are a set of "contexts", the
container is just one of them. Like selectables but simpler, providing
only "wakeup" not read/write. Each connection, listener and the
container have a context, we're thinking about how to expose the
concept so the user can create contexts of their own for syncing
application data structures.

> 
> On Fri, Mar 24, 2017 at 3:30 PM, Alan Conway <acon...@redhat.com>
> wrote:
> 
> > On Fri, 2017-03-24 at 14:06 +0530, Venkat B wrote:
> > > Good day,
> > > 
> > > -- Forwarded message --
> > > > From: Alan Conway <acon...@redhat.com>
> > > > To: users@qpid.apache.org
> > > > Cc:
> > > > Bcc:
> > > > Date: Wed, 22 Mar 2017 14:09:00 -0400
> > > > Subject: Re: [qpid-proton-cpp] default_container does not
> > > > implement
> > > > thread
> > > > safety at all?
> > > > 
> > > 
> > > [snip]
> > > > 
> > > > And to be clear this is abuse to hack out of a short-term hole
> > > > - we
> > > > will have a properly behaved solution soon.
> > > > 
> > > > 
> > > 
> > > Do you have something that I could test?
> > > The code in examples/cpp/mt does not compile out of the box on
> > > 0.17.0
> > > and
> > > in case it is being redesigned anyway then I would prefer to work
> > > with the
> > > new API.
> > > 
> > 
> > I defer to astitcher on the current state of the examples. The API
> > isn't changing significantly, the code in mt/broker.cpp will still
> > be
> > correct with only minor changes if any.
> > 
> > The code in epoll_container.cpp will also still be basically
> > unchanged
> > but writing a custom container will no longer be necessary for most
> > cases: the default_container will be thread-safe and we will have
> > epoll-native, iocp-native and libuv flavours to cover most needs.
> > 
> > 
> > 
> > -
> > 
> > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> > For additional commands, e-mail: users-h...@qpid.apache.org
> > 
> > 


-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: [qpid-proton-cpp] default_container does not implement thread safety at all?

[Fj]

By the way, if anyone is interested, as a stop-gap solution I ended up just
implementing a timer-based callback reenqueuing itself every 50ms and
pulling callbacks from a properly synchronized queue and executing them on
the worker thread. It's not even that bad performance-wise, it just adds
those 50/2 ms of latency on average.

A better solution would be to do the same thing the Python binding does,
expose a Selectable interface and use it to wake up the reactor thread.

And by the way there's a thing that you might want to consider: there's a
bunch of use cases that don't need inter-thread synchronization, for
example, single-threaded workers, or multi-threaded workers that only need
to know when to stop and they know it from the socket being closed. In
those cases exposing a thread-safe Container.inject(callback) is a total
waste, because it means that everything else must constantly take locks for
no reason.

Maybe in the spirit of "you don't pay for what you don't use" the way
Python binding does it is actually fundamentally better, with an
EventInjector thingie that you can add to your Container if you want and
then call injector.inject(callback) instead of container.inject?

On Fri, Mar 24, 2017 at 3:30 PM, Alan Conway <acon...@redhat.com> wrote:

> On Fri, 2017-03-24 at 14:06 +0530, Venkat B wrote:
> > Good day,
> >
> > -- Forwarded message --
> > > From: Alan Conway <acon...@redhat.com>
> > > To: users@qpid.apache.org
> > > Cc:
> > > Bcc:
> > > Date: Wed, 22 Mar 2017 14:09:00 -0400
> > > Subject: Re: [qpid-proton-cpp] default_container does not implement
> > > thread
> > > safety at all?
> > >
> >
> > [snip]
> > >
> > > And to be clear this is abuse to hack out of a short-term hole - we
> > > will have a properly behaved solution soon.
> > >
> > >
> >
> > Do you have something that I could test?
> > The code in examples/cpp/mt does not compile out of the box on 0.17.0
> > and
> > in case it is being redesigned anyway then I would prefer to work
> > with the
> > new API.
> >
>
> I defer to astitcher on the current state of the examples. The API
> isn't changing significantly, the code in mt/broker.cpp will still be
> correct with only minor changes if any.
>
> The code in epoll_container.cpp will also still be basically unchanged
> but writing a custom container will no longer be necessary for most
> cases: the default_container will be thread-safe and we will have
> epoll-native, iocp-native and libuv flavours to cover most needs.
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> For additional commands, e-mail: users-h...@qpid.apache.org
>
>

Re: [qpid-proton-cpp] default_container does not implement thread safety at all?

[Alan Conway]

On Fri, 2017-03-24 at 14:06 +0530, Venkat B wrote:
> Good day,
> 
> -- Forwarded message --
> > From: Alan Conway <acon...@redhat.com>
> > To: users@qpid.apache.org
> > Cc:
> > Bcc:
> > Date: Wed, 22 Mar 2017 14:09:00 -0400
> > Subject: Re: [qpid-proton-cpp] default_container does not implement
> > thread
> > safety at all?
> > 
> 
> [snip]
> > 
> > And to be clear this is abuse to hack out of a short-term hole - we
> > will have a properly behaved solution soon.
> > 
> > 
> 
> Do you have something that I could test?
> The code in examples/cpp/mt does not compile out of the box on 0.17.0
> and
> in case it is being redesigned anyway then I would prefer to work
> with the
> new API.
> 

I defer to astitcher on the current state of the examples. The API
isn't changing significantly, the code in mt/broker.cpp will still be 
correct with only minor changes if any. 

The code in epoll_container.cpp will also still be basically unchanged
but writing a custom container will no longer be necessary for most
cases: the default_container will be thread-safe and we will have
epoll-native, iocp-native and libuv flavours to cover most needs.



-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: [qpid-proton-cpp] default_container does not implement thread safety at all?

[Venkat B]

Good day,

-- Forwarded message --
> From: Alan Conway <acon...@redhat.com>
> To: users@qpid.apache.org
> Cc:
> Bcc:
> Date: Wed, 22 Mar 2017 14:09:00 -0400
> Subject: Re: [qpid-proton-cpp] default_container does not implement thread
> safety at all?
>
[snip]
>
> And to be clear this is abuse to hack out of a short-term hole - we
> will have a properly behaved solution soon.
>
>
Do you have something that I could test?
The code in examples/cpp/mt does not compile out of the box on 0.17.0 and
in case it is being redesigned anyway then I would prefer to work with the
new API.

Regards,
Venkat

Re: [qpid-proton-cpp] default_container does not implement thread safety at all?

[Alan Conway]

On Wed, 2017-03-15 at 22:32 +, Adel Boutros wrote:
> Hello,
> 
> As Alan said, we have for example "abused" the schedule in C++. Today
> we run the container on another thread and we have a shared queue
> between the container threads and the other threads. We call schedule
> very frequently with a delay of 0. Whenever the container thread
> wakes, it checks the queue to send events then calls schedule again.
> The access to the queue is protected by locks.
> 
> The downside is that the container thread abuses a full CPU but we
> can limit this by using a delay of higher than 0 on schedule.
> 
> Regards,
> Adel
> 

And to be clear this is abuse to hack out of a short-term hole - we
will have a properly behaved solution soon.

> 
> From: Alan Conway <acon...@redhat.com>
> Sent: Wednesday, March 15, 2017 10:51:24 PM
> To: users@qpid.apache.org
> Subject: Re: [qpid-proton-cpp] default_container does not implement
> thread safety at all?
> 
> On Tue, 2017-03-14 at 01:23 +0200, Fj wrote:
> > Hello, I'm mightily confused.
> > 
> > There's
> > https://qpid.apache.org/releases/qpid-proton-0.17.0/proton/cpp/api/
> > md
> > _mt.html
> > that says in no uncertain terms that I can use event_loop.inject()
> > to
> > execute a callback on the thread that runs the event loop. There's
> > an
> > examples/cpp/mt/broker.cpp linked from the above that's supposed to
> > be a
> > thread safe implementation of a thread pool running a bunch of
> > containers.
> > 
> > But after "grep inject" and carefully sifting through the results,
> > it
> > seems
> > that after some indirection it all ends up calling
> > 
> > // FIXME aconway 2016-06-07: this is not thread safe. It is
> > sufficient for
> > using
> > // default_container::schedule() inside a handler but not for
> > inject() from
> > // another thread.
> > bool event_loop::impl::inject(void_function0& f) {
> > try { f(); } catch(...) {}
> > return true;
> > }
> > 
> > What the hell? Blank exception suppression is just the icing on the
> > cake,
> > nothing here even resembles doing the actual hard thing: telling
> > the
> > reactor to inject a new custom event in a threadsafe manner.
> 
> Guilty as charged mlud. This was work in progress when we started to
> redo the guts in a more profound way. The Real Thing is coming - it
> is
> available in C now as pn_proactor and the C++ binding is being
> refactored on top of it.
> 
> > Am I missing something obvious, or is the documentation there, and
> > in
> > other
> > places, and the example, chemically pure wishful thinking, like, it
> > would
> > be pretty nice if we supported multithreading, and it would work in
> > such
> > and such ways then, but unfortunately we don't, as a matter of
> > fact?
> 
> The state of the release is probably not clearly documented: the C++
> MT
> interfaces *allow* you to build an MT app that replaces the
> proton::container (as demoed in the mt_broker) BUT the
> proton::container itself is still not multithreaded. It has a
> compatible API because it will be in future.
> 
> > If so, maybe you gals and guys should fix the documentation, and
> > not
> > just
> > with "experimental" but with "DOESN'T WORK AT ALL" prominent on the
> > page?
> 
> I agree the doc is not clear, next release it should work as expected
> an not as obscurely caveatted.
> 
> > On more constructive notes:
> > 
> > 1) do people maybe use some custom containers, similar to the
> > epoll_container in the examples (which doesn't support schedule()
> > though, I
> > have to point out)? If so, I much desire to see one.
> 
> Yes, but the plan is to fix the non-custom container to be fully
> thread-safe so you don't have to unless you have special needs. You
> can
> also implement directly in C if you have very special needs.
> 
> > 2) why do you attempt to bolt on your event_loop thing to
> > Connection
> > and
> > below when the only thing that has the run() method is Container,
> > therefore
> > that's the scope that any worker thread would have? It's the
> > Container that
> > should have the inject() method. Underlying C API notwithstanding.
> 
> Nope: the threading model is seralized-per-connection, so functions
> that operate only on connection state and are triggered from an event
> handler don't need to be locked. Only interactions that cross
> connections (e.g. one connection queues something a s

Re: [qpid-proton-cpp] default_container does not implement thread safety at all?

[Adel Boutros]

Hello,

As Alan said, we have for example "abused" the schedule in C++. Today we run 
the container on another thread and we have a shared queue between the 
container threads and the other threads. We call schedule very frequently with 
a delay of 0. Whenever the container thread wakes, it checks the queue to send 
events then calls schedule again. The access to the queue is protected by locks.

The downside is that the container thread abuses a full CPU but we can limit 
this by using a delay of higher than 0 on schedule.

Regards,
Adel


From: Alan Conway <acon...@redhat.com>
Sent: Wednesday, March 15, 2017 10:51:24 PM
To: users@qpid.apache.org
Subject: Re: [qpid-proton-cpp] default_container does not implement thread 
safety at all?

On Tue, 2017-03-14 at 01:23 +0200, Fj wrote:
> Hello, I'm mightily confused.
>
> There's
> https://qpid.apache.org/releases/qpid-proton-0.17.0/proton/cpp/api/md
> _mt.html
> that says in no uncertain terms that I can use event_loop.inject() to
> execute a callback on the thread that runs the event loop. There's an
> examples/cpp/mt/broker.cpp linked from the above that's supposed to
> be a
> thread safe implementation of a thread pool running a bunch of
> containers.
>
> But after "grep inject" and carefully sifting through the results, it
> seems
> that after some indirection it all ends up calling
>
> // FIXME aconway 2016-06-07: this is not thread safe. It is
> sufficient for
> using
> // default_container::schedule() inside a handler but not for
> inject() from
> // another thread.
> bool event_loop::impl::inject(void_function0& f) {
> try { f(); } catch(...) {}
> return true;
> }
>
> What the hell? Blank exception suppression is just the icing on the
> cake,
> nothing here even resembles doing the actual hard thing: telling the
> reactor to inject a new custom event in a threadsafe manner.

Guilty as charged mlud. This was work in progress when we started to
redo the guts in a more profound way. The Real Thing is coming - it is
available in C now as pn_proactor and the C++ binding is being
refactored on top of it.

> Am I missing something obvious, or is the documentation there, and in
> other
> places, and the example, chemically pure wishful thinking, like, it
> would
> be pretty nice if we supported multithreading, and it would work in
> such
> and such ways then, but unfortunately we don't, as a matter of fact?

The state of the release is probably not clearly documented: the C++ MT
interfaces *allow* you to build an MT app that replaces the
proton::container (as demoed in the mt_broker) BUT the
proton::container itself is still not multithreaded. It has a
compatible API because it will be in future.

> If so, maybe you gals and guys should fix the documentation, and not
> just
> with "experimental" but with "DOESN'T WORK AT ALL" prominent on the
> page?

I agree the doc is not clear, next release it should work as expected
an not as obscurely caveatted.

> On more constructive notes:
>
> 1) do people maybe use some custom containers, similar to the
> epoll_container in the examples (which doesn't support schedule()
> though, I
> have to point out)? If so, I much desire to see one.

Yes, but the plan is to fix the non-custom container to be fully
thread-safe so you don't have to unless you have special needs. You can
also implement directly in C if you have very special needs.

> 2) why do you attempt to bolt on your event_loop thing to Connection
> and
> below when the only thing that has the run() method is Container,
> therefore
> that's the scope that any worker thread would have? It's the
> Container that
> should have the inject() method. Underlying C API notwithstanding.

Nope: the threading model is seralized-per-connection, so functions
that operate only on connection state and are triggered from an event
handler don't need to be locked. Only interactions that cross
connections (e.g. one connection queues something a subscriber on
another connection needs to wake up) need sync. This is demonstrated
with the mt_broker stuff. That's why injecting is per-connection.

> 3) What's the best way forward if I really have to have a threadsafe
> container that I can inject some event into threadsafely:

The example demonstrates how you can do it, but is incomplete as you
point out. Soon it will be built in.

>   3a) does the C API support thread safety? Like, the whole problem
> is
> touching the reactor's job queue or whatever it has with taking an
> appropriate lock, and it must take that same lock itself for it to
> work.

It does now, see the pn_proactor, there are examples. The C reactor is
fundamentally and forever thread-unsafe and horribly broken in all ways
with respe

Re: [qpid-proton-cpp] default_container does not implement thread safety at all?

[Alan Conway]

On Tue, 2017-03-14 at 01:23 +0200, Fj wrote:
> Hello, I'm mightily confused.
> 
> There's
> https://qpid.apache.org/releases/qpid-proton-0.17.0/proton/cpp/api/md
> _mt.html
> that says in no uncertain terms that I can use event_loop.inject() to
> execute a callback on the thread that runs the event loop. There's an
> examples/cpp/mt/broker.cpp linked from the above that's supposed to
> be a
> thread safe implementation of a thread pool running a bunch of
> containers.
> 
> But after "grep inject" and carefully sifting through the results, it
> seems
> that after some indirection it all ends up calling
> 
> // FIXME aconway 2016-06-07: this is not thread safe. It is
> sufficient for
> using
> // default_container::schedule() inside a handler but not for
> inject() from
> // another thread.
> bool event_loop::impl::inject(void_function0& f) {
> try { f(); } catch(...) {}
> return true;
> }
> 
> What the hell? Blank exception suppression is just the icing on the
> cake,
> nothing here even resembles doing the actual hard thing: telling the
> reactor to inject a new custom event in a threadsafe manner.

Guilty as charged mlud. This was work in progress when we started to
redo the guts in a more profound way. The Real Thing is coming - it is
available in C now as pn_proactor and the C++ binding is being
refactored on top of it.

> Am I missing something obvious, or is the documentation there, and in
> other
> places, and the example, chemically pure wishful thinking, like, it
> would
> be pretty nice if we supported multithreading, and it would work in
> such
> and such ways then, but unfortunately we don't, as a matter of fact?

The state of the release is probably not clearly documented: the C++ MT
interfaces *allow* you to build an MT app that replaces the
proton::container (as demoed in the mt_broker) BUT the
proton::container itself is still not multithreaded. It has a
compatible API because it will be in future.

> If so, maybe you gals and guys should fix the documentation, and not
> just
> with "experimental" but with "DOESN'T WORK AT ALL" prominent on the
> page?

I agree the doc is not clear, next release it should work as expected
an not as obscurely caveatted.

> On more constructive notes:
> 
> 1) do people maybe use some custom containers, similar to the
> epoll_container in the examples (which doesn't support schedule()
> though, I
> have to point out)? If so, I much desire to see one.

Yes, but the plan is to fix the non-custom container to be fully
thread-safe so you don't have to unless you have special needs. You can
also implement directly in C if you have very special needs.

> 2) why do you attempt to bolt on your event_loop thing to Connection
> and
> below when the only thing that has the run() method is Container,
> therefore
> that's the scope that any worker thread would have? It's the
> Container that
> should have the inject() method. Underlying C API notwithstanding.

Nope: the threading model is seralized-per-connection, so functions
that operate only on connection state and are triggered from an event
handler don't need to be locked. Only interactions that cross
connections (e.g. one connection queues something a subscriber on
another connection needs to wake up) need sync. This is demonstrated
with the mt_broker stuff. That's why injecting is per-connection.

> 3) What's the best way forward if I really have to have a threadsafe
> container that I can inject some event into threadsafely:

The example demonstrates how you can do it, but is incomplete as you
point out. Soon it will be built in.

>   3a) does the C API support thread safety? Like, the whole problem
> is
> touching the reactor's job queue or whatever it has with taking an
> appropriate lock, and it must take that same lock itself for it to
> work.

It does now, see the pn_proactor, there are examples. The C reactor is
fundamentally and forever thread-unsafe and horribly broken in all ways
with respect to concurrency. We started to introduce MT at the C++
layer and work around the reactor - proton::container lagged because it
is so tied to the reactor. 

Finally we decided to replace the pn_reactor entirely with the
pn_proactor in C - this is designed to be concurrent and provide for
replaceable IO. At this point the proactor exists and has initial
implementations (libuv done, native iocp + epoll nearly so) but we are
still working on replacing the reactor. It's been a big job.

> 
>   3b) I checked out the Python API, they use a mock EventInjector
> pollable
> thingie instead of reaching inside the reactor and telling it add an
> event.

Yep, the python API is due for an overhaul also but works for now as
far as python threads are concerned. Ruby likewise. 

>   3c) epoll example does yet another thing apparently (though I'm not
> sure
> it works because it's not covered by any tests): just tell the
> reactor to
> wake up and process the queued job list in your handler.

The reactor is a dead-end threading 

Re: [qpid-proton-cpp] default_container does not implement thread safety at all?

[Andrew Stitcher]

On Tue, 2017-03-14 at 01:23 +0200, Fj wrote:
> ... (elided somewhat inflammatory comments)

> Am I missing something obvious, 

That's difficult to say (obvious is so subjective!)

The code you are looking at implements the multithreaded API, but is
not multithreaded itself, and yes it is unfortunate that it doesn't do
a thread safe inject, but C++03 has no threading capability and the st
code will compile with 03. Since it's important for us to minimise
dependencies for the proton libraries, this code can't do much better.

In the same code drop you will find multithreaded implementations of
the container API too - This is in the examples/cpp/mt directory, this
code relies on external thread capabilities.

Note that we use the threading abilities of the C++11 and later
standard library, so you will need to compile with a modern compiler to
get mt support.

Note that this does mean that if you want to support *any* variant of
multithreading with the C++ binding (either multiple worker threads for
the container, or running the container in a separate thread to the
rest of your application) you will need C++11 or later to compile the
proton C++ binding library. I don't consider this to be a real problem
at this point. C++11 is supported well enough for our purposes in all
popular C++ compilers, and has been for quite some time - I think over
5 years.

> or is the documentation there, and in
> other
> places, and the example, chemically pure wishful thinking, like, it
> would
> be pretty nice if we supported multithreading, and it would work in
> such
> and such ways then, but unfortunately we don't, as a matter of fact?
> 
> If so, maybe you gals and guys should fix the documentation, and not
> just
> with "experimental" but with "DOESN'T WORK AT ALL" prominent on the
> page?

Experimental means that the API is not stable, but is provided so you
can get something done, and hopefully let us know how it goes, and give
us feedback so we can stabilise the API so that it can be released as
no longer experimental.

All this code is being replaced as we speak with a proactor based
implementation of the container API; this should support multiple
worker threads in the container as long as you have C++11 or later. but
only a single thread with C++03.

> On more constructive notes:
> 
> 1) do people maybe use some custom containers, similar to the
> epoll_container in the examples (which doesn't support schedule()
> though, I
> have to point out)? If so, I much desire to see one.

Custom containers essentially go away in the latest version of the API.
Instead of suggesting the user might want to implement their own
container we are providing a proactor based container.

We are providing proactor implementations based in libuv, epoll (for
Linux), iocp (for Windows). It will be possible to write your own
proactor if necessary, but the ones we will provide cover a lot of the
usual implementation landscape.

> 
> 2) why do you attempt to bolt on your event_loop thing to Connection
> and
> below when the only thing that has the run() method is Container,
> therefore
> that's the scope that any worker thread would have? It's the
> Container that
> should have the inject() method. Underlying C API notwithstanding.

I think you may have a misunderstanding here. Although the run loops
are owned by the container, the unit of serialisation is the
connection. The "bolting on" is to give access from whichever object
you have to a thread safe context to inject work. Even if the inject
happens at the container level, it's convenient to be able to inject
from an event handler based on the object contained in the event,
rather than having to navigate up to the container and then injecting. 
You *must* specify the connection that is to serialised, as
*everything* happens per connection, so there is no value in running
code in the run loop without having some way to safely run code in the
connection context.

The container schedule API does run code only in the container context,
with no connection serialisation, but it's not safe to use to refer to
any connection (or lower ) state without injecting into the relevant
connection.

The "event_loop" concept is to indicate the unit of serialisation. In
theory this could be any level of the object tree. I agree that
"event_loop" is a somewhat confusing name to indicate the serialisation
unit, but we couldn't think of a better one - probably we need to make
the doc clearer.


> 
> 3) What's the best way forward if I really have to have a threadsafe
> container that I can inject some event into threadsafely:

For immediate trying the example mt container is probably your best
bet. In the near future the proactor based container is "the future"
(tm).

> 
>   3a) does the C API support thread safety? Like, the whole problem
> is
> touching the reactor's job queue or whatever it has with taking an
> appropriate lock, and it must take that same lock itself for it to
> work.

I don't think the C 

[qpid-proton-cpp] default_container does not implement thread safety at all?

[Fj]

Hello, I'm mightily confused.

There's
https://qpid.apache.org/releases/qpid-proton-0.17.0/proton/cpp/api/md_mt.html
that says in no uncertain terms that I can use event_loop.inject() to
execute a callback on the thread that runs the event loop. There's an
examples/cpp/mt/broker.cpp linked from the above that's supposed to be a
thread safe implementation of a thread pool running a bunch of containers.

But after "grep inject" and carefully sifting through the results, it seems
that after some indirection it all ends up calling

// FIXME aconway 2016-06-07: this is not thread safe. It is sufficient for
using
// default_container::schedule() inside a handler but not for inject() from
// another thread.
bool event_loop::impl::inject(void_function0& f) {
try { f(); } catch(...) {}
return true;
}

What the hell? Blank exception suppression is just the icing on the cake,
nothing here even resembles doing the actual hard thing: telling the
reactor to inject a new custom event in a threadsafe manner.

Am I missing something obvious, or is the documentation there, and in other
places, and the example, chemically pure wishful thinking, like, it would
be pretty nice if we supported multithreading, and it would work in such
and such ways then, but unfortunately we don't, as a matter of fact?

If so, maybe you gals and guys should fix the documentation, and not just
with "experimental" but with "DOESN'T WORK AT ALL" prominent on the page?



On more constructive notes:

1) do people maybe use some custom containers, similar to the
epoll_container in the examples (which doesn't support schedule() though, I
have to point out)? If so, I much desire to see one.

2) why do you attempt to bolt on your event_loop thing to Connection and
below when the only thing that has the run() method is Container, therefore
that's the scope that any worker thread would have? It's the Container that
should have the inject() method. Underlying C API notwithstanding.

3) What's the best way forward if I really have to have a threadsafe
container that I can inject some event into threadsafely:

  3a) does the C API support thread safety? Like, the whole problem is
touching the reactor's job queue or whatever it has with taking an
appropriate lock, and it must take that same lock itself for it to work.

  3b) I checked out the Python API, they use a mock EventInjector pollable
thingie instead of reaching inside the reactor and telling it add an event.

  3c) epoll example does yet another thing apparently (though I'm not sure
it works because it's not covered by any tests): just tell the reactor to
wake up and process the queued job list in your handler.