Re: [asterisk-users] Which CDR processing for high load ?

2018-02-22 Thread Richard Mudgett 
On Thu, Feb 22, 2018 at 3:23 PM, Olivier  wrote:

>
> 1. Would say CDR_ODBC has a greater chance than CDR_CUSTOM (if I may call
> them both as such) to become a bottleneck under pressure ?
>

I don't know.  The cdr_custom back end appends CSV records to the end of a
text file you specify.  The cdr_odbc back end sends the records to a
database.


> 2. I didn't know about batch mode existence. Thanks for presenting it.
> In such batch mode, where are stored CDRs before being written to files or
> database ? Does it also apply to data later found in cdr-scv/Master.csv
> file ?
>

Before CDRs get written to the back ends (i.e., permanent storage) they are
in memory data structures.  Where else could they be before getting written
to the back ends?

Richard
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
  https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

Re: [asterisk-users] Which CDR processing for high load ?

2018-02-22 Thread Khalil Khamlichi 
what kind of ippbx does 50cps ? I think you must be looking for something
like opensips.

On Feb 22, 2018 9:13 PM, "Olivier"  wrote:



2018-02-22 17:12 GMT+01:00 Dovid Bender :

> Have you looked at the proc limits for the Asterisk PID?
>
>
>
No I haven't.
What shall I look for, exactly ?

The only thing I configured for performance is asterisk.conf 's maxfiles
parameter.

--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.
org/

New to Asterisk? Start here:
  https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
  https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

Re: [asterisk-users] Which CDR processing for high load ?

2018-02-22 Thread Olivier 
1. Would say CDR_ODBC has a greater chance than CDR_CUSTOM (if I may call
them both as such) to become a bottleneck under pressure ?

2. I didn't know about batch mode existence. Thanks for presenting it.
In such batch mode, where are stored CDRs before being written to files or
database ? Does it also apply to data later found in cdr-scv/Master.csv
file ?

2018-02-22 18:14 GMT+01:00 Richard Mudgett :

>
>
> On Thu, Feb 22, 2018 at 5:23 AM, Olivier  wrote:
>
>> Hello,
>>
>> I'm load testing a new Asterisk 13 system (Debian  Stretch, packaged
>> asterisk).
>> One system writes CDR though an ODBC connection to a local Postgres
>> database over the LAN.
>>
>>
>> When sending 50 new calls per second with SIPp, I'm seeing one system
>> outputs :
>> taskprocessor.c: The 'subm:cdr_engine-0003' task processor queue
>> reached 5000 scheduled tasks again.
>>
>> This [1] thread mentions such limit.
>>
>> I was thinking of simply writing CDR entries to a local file before
>> centralizing them into my database.
>>
>> What would you suggest on this topic ?
>>
>
> You should get better CDR performance enabling batch mode which is
> disabled by default for legacy reasons.
>
> The CDR code processes all call events in one thread to create CDRs.  This
> thread is what processes the
> 'subm:cdr_engine-0003' task processor queue you mention above.  If you
> don't have batch mode enabled
> then that same thread also has to immediately write the CDRs to each back
> end configured.  If you do have
> batch mode enabled then the CDRs are passed to another thread to write to
> the back ends.
>
> You need to be using at least v13.19.1 or v15.2.1 to also have some CDR
> performance enhancements to
> help CDR processing of call events.
>
> For information about ODBC connection pooling performance problems see [2].
>
> Richard
>
> [2] http://blogs.asterisk.org/2016/06/15/asterisk-odbc-connections/
>
>
>>
>>
>> Best regards
>>
>> [1] http://lists.digium.com/pipermail/asterisk-dev/2016-June/075607.html
>>
>> --
>> _
>> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>>
>> Check out the new Asterisk community forum at:
>> https://community.asterisk.org/
>>
>> New to Asterisk? Start here:
>>   https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>>
>> asterisk-users mailing list
>> To UNSUBSCRIBE or update options visit:
>>http://lists.digium.com/mailman/listinfo/asterisk-users
>>
>
>
> --
> _
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> Check out the new Asterisk community forum at: https://community.asterisk.
> org/
>
> New to Asterisk? Start here:
>   https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>http://lists.digium.com/mailman/listinfo/asterisk-users
>
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
  https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

Re: [asterisk-users] Which CDR processing for high load ?

2018-02-22 Thread Olivier 
2018-02-22 17:12 GMT+01:00 Dovid Bender :

> Have you looked at the proc limits for the Asterisk PID?
>
>
>
No I haven't.
What shall I look for, exactly ?

The only thing I configured for performance is asterisk.conf 's maxfiles
parameter.
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
  https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

Re: [asterisk-users] Which CDR processing for high load ?

2018-02-22 Thread Richard Mudgett 
On Thu, Feb 22, 2018 at 5:23 AM, Olivier  wrote:

> Hello,
>
> I'm load testing a new Asterisk 13 system (Debian  Stretch, packaged
> asterisk).
> One system writes CDR though an ODBC connection to a local Postgres
> database over the LAN.
>
>
> When sending 50 new calls per second with SIPp, I'm seeing one system
> outputs :
> taskprocessor.c: The 'subm:cdr_engine-0003' task processor queue
> reached 5000 scheduled tasks again.
>
> This [1] thread mentions such limit.
>
> I was thinking of simply writing CDR entries to a local file before
> centralizing them into my database.
>
> What would you suggest on this topic ?
>

You should get better CDR performance enabling batch mode which is disabled
by default for legacy reasons.

The CDR code processes all call events in one thread to create CDRs.  This
thread is what processes the
'subm:cdr_engine-0003' task processor queue you mention above.  If you
don't have batch mode enabled
then that same thread also has to immediately write the CDRs to each back
end configured.  If you do have
batch mode enabled then the CDRs are passed to another thread to write to
the back ends.

You need to be using at least v13.19.1 or v15.2.1 to also have some CDR
performance enhancements to
help CDR processing of call events.

For information about ODBC connection pooling performance problems see [2].

Richard

[2] http://blogs.asterisk.org/2016/06/15/asterisk-odbc-connections/


>
>
> Best regards
>
> [1] http://lists.digium.com/pipermail/asterisk-dev/2016-June/075607.html
>
> --
> _
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> Check out the new Asterisk community forum at: https://community.asterisk.
> org/
>
> New to Asterisk? Start here:
>   https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>http://lists.digium.com/mailman/listinfo/asterisk-users
>
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
  https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

Re: [asterisk-users] Which CDR processing for high load ?

2018-02-22 Thread Dovid Bender 
Have you looked at the proc limits for the Asterisk PID?


On Thu, Feb 22, 2018 at 6:23 AM, Olivier  wrote:

> Hello,
>
> I'm load testing a new Asterisk 13 system (Debian  Stretch, packaged
> asterisk).
> One system writes CDR though an ODBC connection to a local Postgres
> database over the LAN.
>
>
> When sending 50 new calls per second with SIPp, I'm seeing one system
> outputs :
> taskprocessor.c: The 'subm:cdr_engine-0003' task processor queue
> reached 5000 scheduled tasks again.
>
> This [1] thread mentions such limit.
>
> I was thinking of simply writing CDR entries to a local file before
> centralizing them into my database.
>
> What would you suggest on this topic ?
>
> Best regards
>
> [1] http://lists.digium.com/pipermail/asterisk-dev/2016-June/075607.html
>
> --
> _
> -- Bandwidth and Colocation Provided by http://www.api-digital.com --
>
> Check out the new Asterisk community forum at: https://community.asterisk.
> org/
>
> New to Asterisk? Start here:
>   https://wiki.asterisk.org/wiki/display/AST/Getting+Started
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>http://lists.digium.com/mailman/listinfo/asterisk-users
>
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
  https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users

[asterisk-users] Which CDR processing for high load ?

2018-02-22 Thread Olivier 
Hello,

I'm load testing a new Asterisk 13 system (Debian  Stretch, packaged
asterisk).
One system writes CDR though an ODBC connection to a local Postgres
database over the LAN.


When sending 50 new calls per second with SIPp, I'm seeing one system
outputs :
taskprocessor.c: The 'subm:cdr_engine-0003' task processor queue
reached 5000 scheduled tasks again.

This [1] thread mentions such limit.

I was thinking of simply writing CDR entries to a local file before
centralizing them into my database.

What would you suggest on this topic ?

Best regards

[1] http://lists.digium.com/pipermail/asterisk-dev/2016-June/075607.html
-- 
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --

Check out the new Asterisk community forum at: https://community.asterisk.org/

New to Asterisk? Start here:
  https://wiki.asterisk.org/wiki/display/AST/Getting+Started

asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users