Application crash in Qpid proton C++ pn_class_incref function

[Arun Koshal]

Hi,

We are using Apache Qpid Proton C++ library version 0.37 for posting logs
to Azure eventhub in my application. Recently we have seen some crashes in
the field which point to the pn_class_incref() function in Apache Qpid
Proton library. We could not get the Core dump file but we have got the
call stack which looks like below:

0# switch_signal_handler(int) in ./Switch
1# 0x00007F6A401CD520 in /lib/x86_64-linux-gnu/libc.so.6
2# pn_class_incref in /lib/libqpid-proton-core.so.10
3# pn_class_incref in /lib/libqpid-proton-core.so.10
4# proton::sender::send(proton::message const&, proton::binary const&) in
/lib/libqpid-proton-cpp.so.12
5# proton::sender::send(proton::message const&) in
/lib/libqpid-proton-cpp.so.12
6# LogUploaderClient::post_log_to_eventhub(char const*) in ./Switch
7# LogUploaderClient::process_new_log(char*) in ./Switch
8# LogUploaderClient::log_uploader_thread() in ./Switch
9# 0x00007F6A4048F2B3 in /lib/x86_64-linux-gnu/libstdc++.so.6
10# 0x00007F6A4021FB43 in /lib/x86_64-linux-gnu/libc.so.6
11# 0x00007F6A402B1A00 in /lib/x86_64-linux-gnu/libc.so.6

Please find below the implementation of the send method in application:
        bool send(std::string msg)
        {
            if ((msg.size() > 0) && (sendable_) && (sender_.credit() > 0))
            {
                proton::binary bodyBinary(msg.begin(), msg.end());
                proton::message m;
                m.body(bodyBinary);
                m.inferred(true);
                sender_.send(m);
                messages_sent_++;
                return true;
            }

            return false;
        }

Please note that sender_ is an object of proton::sender class.
Please find below the implementation of post_log_to_eventhub() method which
invokes this send method:

void LogUploaderClient::post_log_to_eventhub(const char* buf)
{
    if (log_eventhub_sender_)
    {
        const int32_t max_retries { 15 };
        const int32_t duration { 1 }; //second
        int32_t retry { 0 };

        while (!log_eventhub_sender_->ready())
        {
            retry++;
            if (retry > max_retries)
                break;
            usleep(duration * 1000 * 1000);
        }

        if (log_eventhub_sender_->ready())
        {
            log_eventhub_sender_->send(buf);
        }
    }
}

Unfortunately, we do not see this type of exception in our local lab
environment but it has been observed quite consistently in the field.

Any clue or help to diagnose and resolve this problem would really be
great. Please note that we have this signal handler (switch_signal_handler
function) registered for SIGINT, SIGSEGV and SIGABRT.

Thank you so much!

Best regards,
Arun