Re: [cisco-voip] Automated PSTN ingress call regression testing?

[Stephen Welsh]

Hi Nick,

We (UnifiedFX) are exposing AXL/JTAPI and a few other CUCM API’s via a simple 
REST API and have created a Python SDK to make it easy to get up and running.

AutomationFX Overview and placeholder for further libraries/docs:
https://github.com/unifiedfx/awesome-automationfx

AutomationFX Python SDK
https://github.com/unifiedfx/automationfx-python

So as long as you have some spare IP Phones you can perform automated calling 
with a simple Python script (Example 
Tests).
 It supports multi-cluster (including mixed CUCM versions) so you could get 
some test trunks to septette CUCM cluster with some ‘PSTN’ phones and 
co-ordinate calls from the PSTN to on-net, answer/hold/transfer etc.

Obviously scaling up is more of a problem, currently you need real phones, but 
at some point we shall provide CTI Ports too ;)

We have not released AutomationFX for public download yet, happy to let you 
give it a spin if you like.

Kind Regards

Stephen Welsh
CTO
UnifiedFX


On 8 Mar 2018, at 20:13, Nick Barnett 
> wrote:

Thanks. I am aware of the multiple carriers... but I think being able to cover 
the edge services would be a huge help in this situation. I'm just anticipating 
what they will ask for. I kind of cast a wide net with this email because I was 
not sure what kind of services were out there. If some provider offered a 
prepackaged, automated testing service that featured multiple carrier numbers, 
I'd buy it in an instant. I just need to remember baby steps :)

By "prone to issues", I just mean that it will test connected calls, but 
getting that next layer of "it connected, but is it working" would be 
difficult. Not necessarily "issues". I suppose "obstacles" would have been a 
better word.  The issues I see with the freePBX would be similar, but also 
includes perimeter security and things of that nature.

I don't have UCCX, but I'm fairly ok at cobbling together AXL and JTAPI to do 
some stuff... maybe I'll just start there since it's basically free.

On Wed, Mar 7, 2018 at 9:33 PM, Anthony Holloway 
> wrote:
Even if you do the Free IP PBX or Twilio API, you're only calling from one 
carrier.  In the scenario you described, you mentioned:

"Verizon wireless customers cannot call Sprint toll free numbers from area code 
555"

Which is very specific.  Would you imagine that you would have owned a 555 
number on Verizon to have caught that scenario faster?  What if the area code 
was 666?  Or the originating carrier was AT?  The different combinations you 
would have to account for are very high.

If you only care about your edge service and inward, and not far end carriers, 
then a Twilio API app sounds like a good plan.  Heck, you could even just write 
a UCCX script to call out and back in via tromboning off the PSTN.

I'm curious, what did you mean by "prone to issues," when referring to the API?

On Wed, Mar 7, 2018 at 1:57 PM Nick Barnett 
> wrote:
A client has a need for an off site solution that will make test calls to their 
numbers and report when there are issues. I understand that this is very vague, 
but they are interested in hearing about any and all solutions.

They have several SIP carriers and a nationwide presence, but the SIP trunking 
is centralized. They've had enough issues with one DID service failing and 
their customers having to report the issue. Ideally, the SIP providers would be 
able to automatically do "something" when they stop receiving options pings, or 
when a certain sip response is received... but it doesn't work that way with 
the behemoth phone companies.

The way it works now is that MOST issues are able to be caught successfully 
with internal monitoring... but others such certain NPA-NXX can't call another 
NPA-NXX, or carrier interconnects such as "Verizon wireless customers cannot 
call Sprint toll free numbers from area code 555"  These odd scenarios are what 
we are looking to solve. I understand this is potentially huge, but I think if 
we could automate calls to about 10 different numbers, that would cover enough 
of the ingress and carrier combinations that it would make a HUGE difference.

I've thought of spinning up an Asterisk and somehow automating the echo test 
feature. I've also thought about using the Twilio API to test if calls are 
successful. Both of these are complicated and prone to issues... so if there is 
a hosted or cloud solution that is already available, please let me know.

Any suggestions or more than welcome.

Thanks,
Nick
___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

___
cisco-voip mailing 

Re: [cisco-voip] Automated PSTN ingress call regression testing?

[Loren Hillukka]

Startrinity is a pretty awesome tool. Not free but has pretty amazing 
capabilities.

Loren

On Mar 8, 2018, at 2:17 PM, Wes Sisk (wsisk) 
> wrote:

I have seen Hammer used for inbound call verification before:

http://www.empirix.com/products/hammer/

-Wes

On Mar 8, 2018, at 3:13 PM, Nick Barnett 
> wrote:

Thanks. I am aware of the multiple carriers... but I think being able to cover 
the edge services would be a huge help in this situation. I'm just anticipating 
what they will ask for. I kind of cast a wide net with this email because I was 
not sure what kind of services were out there. If some provider offered a 
prepackaged, automated testing service that featured multiple carrier numbers, 
I'd buy it in an instant. I just need to remember baby steps :)

By "prone to issues", I just mean that it will test connected calls, but 
getting that next layer of "it connected, but is it working" would be 
difficult. Not necessarily "issues". I suppose "obstacles" would have been a 
better word.  The issues I see with the freePBX would be similar, but also 
includes perimeter security and things of that nature.

I don't have UCCX, but I'm fairly ok at cobbling together AXL and JTAPI to do 
some stuff... maybe I'll just start there since it's basically free.

On Wed, Mar 7, 2018 at 9:33 PM, Anthony Holloway 
> wrote:
Even if you do the Free IP PBX or Twilio API, you're only calling from one 
carrier.  In the scenario you described, you mentioned:

"Verizon wireless customers cannot call Sprint toll free numbers from area code 
555"

Which is very specific.  Would you imagine that you would have owned a 555 
number on Verizon to have caught that scenario faster?  What if the area code 
was 666?  Or the originating carrier was AT?  The different combinations you 
would have to account for are very high.

If you only care about your edge service and inward, and not far end carriers, 
then a Twilio API app sounds like a good plan.  Heck, you could even just write 
a UCCX script to call out and back in via tromboning off the PSTN.

I'm curious, what did you mean by "prone to issues," when referring to the API?

On Wed, Mar 7, 2018 at 1:57 PM Nick Barnett 
> wrote:
A client has a need for an off site solution that will make test calls to their 
numbers and report when there are issues. I understand that this is very vague, 
but they are interested in hearing about any and all solutions.

They have several SIP carriers and a nationwide presence, but the SIP trunking 
is centralized. They've had enough issues with one DID service failing and 
their customers having to report the issue. Ideally, the SIP providers would be 
able to automatically do "something" when they stop receiving options pings, or 
when a certain sip response is received... but it doesn't work that way with 
the behemoth phone companies.

The way it works now is that MOST issues are able to be caught successfully 
with internal monitoring... but others such certain NPA-NXX can't call another 
NPA-NXX, or carrier interconnects such as "Verizon wireless customers cannot 
call Sprint toll free numbers from area code 555"  These odd scenarios are what 
we are looking to solve. I understand this is potentially huge, but I think if 
we could automate calls to about 10 different numbers, that would cover enough 
of the ingress and carrier combinations that it would make a HUGE difference.

I've thought of spinning up an Asterisk and somehow automating the echo test 
feature. I've also thought about using the Twilio API to test if calls are 
successful. Both of these are complicated and prone to issues... so if there is 
a hosted or cloud solution that is already available, please let me know.

Any suggestions or more than welcome.

Thanks,
Nick
___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip
___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

Re: [cisco-voip] Automated PSTN ingress call regression testing?

[Wes Sisk (wsisk)]

I have seen Hammer used for inbound call verification before:

http://www.empirix.com/products/hammer/

-Wes

On Mar 8, 2018, at 3:13 PM, Nick Barnett 
> wrote:

Thanks. I am aware of the multiple carriers... but I think being able to cover 
the edge services would be a huge help in this situation. I'm just anticipating 
what they will ask for. I kind of cast a wide net with this email because I was 
not sure what kind of services were out there. If some provider offered a 
prepackaged, automated testing service that featured multiple carrier numbers, 
I'd buy it in an instant. I just need to remember baby steps :)

By "prone to issues", I just mean that it will test connected calls, but 
getting that next layer of "it connected, but is it working" would be 
difficult. Not necessarily "issues". I suppose "obstacles" would have been a 
better word.  The issues I see with the freePBX would be similar, but also 
includes perimeter security and things of that nature.

I don't have UCCX, but I'm fairly ok at cobbling together AXL and JTAPI to do 
some stuff... maybe I'll just start there since it's basically free.

On Wed, Mar 7, 2018 at 9:33 PM, Anthony Holloway 
> wrote:
Even if you do the Free IP PBX or Twilio API, you're only calling from one 
carrier.  In the scenario you described, you mentioned:

"Verizon wireless customers cannot call Sprint toll free numbers from area code 
555"

Which is very specific.  Would you imagine that you would have owned a 555 
number on Verizon to have caught that scenario faster?  What if the area code 
was 666?  Or the originating carrier was AT?  The different combinations you 
would have to account for are very high.

If you only care about your edge service and inward, and not far end carriers, 
then a Twilio API app sounds like a good plan.  Heck, you could even just write 
a UCCX script to call out and back in via tromboning off the PSTN.

I'm curious, what did you mean by "prone to issues," when referring to the API?

On Wed, Mar 7, 2018 at 1:57 PM Nick Barnett 
> wrote:
A client has a need for an off site solution that will make test calls to their 
numbers and report when there are issues. I understand that this is very vague, 
but they are interested in hearing about any and all solutions.

They have several SIP carriers and a nationwide presence, but the SIP trunking 
is centralized. They've had enough issues with one DID service failing and 
their customers having to report the issue. Ideally, the SIP providers would be 
able to automatically do "something" when they stop receiving options pings, or 
when a certain sip response is received... but it doesn't work that way with 
the behemoth phone companies.

The way it works now is that MOST issues are able to be caught successfully 
with internal monitoring... but others such certain NPA-NXX can't call another 
NPA-NXX, or carrier interconnects such as "Verizon wireless customers cannot 
call Sprint toll free numbers from area code 555"  These odd scenarios are what 
we are looking to solve. I understand this is potentially huge, but I think if 
we could automate calls to about 10 different numbers, that would cover enough 
of the ingress and carrier combinations that it would make a HUGE difference.

I've thought of spinning up an Asterisk and somehow automating the echo test 
feature. I've also thought about using the Twilio API to test if calls are 
successful. Both of these are complicated and prone to issues... so if there is 
a hosted or cloud solution that is already available, please let me know.

Any suggestions or more than welcome.

Thanks,
Nick
___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

Re: [cisco-voip] Automated PSTN ingress call regression testing?

[Nick Barnett]

Thanks. I am aware of the multiple carriers... but I think being able to
cover the edge services would be a huge help in this situation. I'm just
anticipating what they will ask for. I kind of cast a wide net with this
email because I was not sure what kind of services were out there. If some
provider offered a prepackaged, automated testing service that featured
multiple carrier numbers, I'd buy it in an instant. I just need to remember
baby steps :)

By "prone to issues", I just mean that it will test connected calls, but
getting that next layer of "it connected, but is it working" would be
difficult. Not necessarily "issues". I suppose "obstacles" would have been
a better word.  The issues I see with the freePBX would be similar, but
also includes perimeter security and things of that nature.

I don't have UCCX, but I'm fairly ok at cobbling together AXL and JTAPI to
do some stuff... maybe I'll just start there since it's basically free.

On Wed, Mar 7, 2018 at 9:33 PM, Anthony Holloway <
avholloway+cisco-v...@gmail.com> wrote:

> Even if you do the Free IP PBX or Twilio API, you're only calling from one
> carrier.  In the scenario you described, you mentioned:
>
> "Verizon wireless customers cannot call Sprint toll free numbers from area
> code 555"
>
> Which is very specific.  Would you imagine that you would have owned a 555
> number on Verizon to have caught that scenario faster?  What if the area
> code was 666?  Or the originating carrier was AT?  The different
> combinations you would have to account for are very high.
>
> If you only care about your edge service and inward, and not far end
> carriers, then a Twilio API app sounds like a good plan.  Heck, you could
> even just write a UCCX script to call out and back in via tromboning off
> the PSTN.
>
> I'm curious, what did you mean by "prone to issues," when referring to the
> API?
>
> On Wed, Mar 7, 2018 at 1:57 PM Nick Barnett 
> wrote:
>
>> A client has a need for an off site solution that will make test calls to
>> their numbers and report when there are issues. I understand that this is
>> very vague, but they are interested in hearing about any and all solutions.
>>
>> They have several SIP carriers and a nationwide presence, but the SIP
>> trunking is centralized. They've had enough issues with one DID service
>> failing and their customers having to report the issue. Ideally, the SIP
>> providers would be able to automatically do "something" when they stop
>> receiving options pings, or when a certain sip response is received... but
>> it doesn't work that way with the behemoth phone companies.
>>
>> The way it works now is that MOST issues are able to be caught
>> successfully with internal monitoring... but others such certain NPA-NXX
>> can't call another NPA-NXX, or carrier interconnects such as "Verizon
>> wireless customers cannot call Sprint toll free numbers from area code
>> 555"  These odd scenarios are what we are looking to solve. I understand
>> this is potentially huge, but I think if we could automate calls to about
>> 10 different numbers, that would cover enough of the ingress and carrier
>> combinations that it would make a HUGE difference.
>>
>> I've thought of spinning up an Asterisk and somehow automating the echo
>> test feature. I've also thought about using the Twilio API to test if calls
>> are successful. Both of these are complicated and prone to issues... so if
>> there is a hosted or cloud solution that is already available, please let
>> me know.
>>
>> Any suggestions or more than welcome.
>>
>> Thanks,
>> Nick
>> ___
>> cisco-voip mailing list
>> cisco-voip@puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-voip
>>
>
___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

Re: [cisco-voip] Automated PSTN ingress call regression testing?

[Anthony Holloway]

Even if you do the Free IP PBX or Twilio API, you're only calling from one
carrier.  In the scenario you described, you mentioned:

"Verizon wireless customers cannot call Sprint toll free numbers from area
code 555"

Which is very specific.  Would you imagine that you would have owned a 555
number on Verizon to have caught that scenario faster?  What if the area
code was 666?  Or the originating carrier was AT?  The different
combinations you would have to account for are very high.

If you only care about your edge service and inward, and not far end
carriers, then a Twilio API app sounds like a good plan.  Heck, you could
even just write a UCCX script to call out and back in via tromboning off
the PSTN.

I'm curious, what did you mean by "prone to issues," when referring to the
API?

On Wed, Mar 7, 2018 at 1:57 PM Nick Barnett  wrote:

> A client has a need for an off site solution that will make test calls to
> their numbers and report when there are issues. I understand that this is
> very vague, but they are interested in hearing about any and all solutions.
>
> They have several SIP carriers and a nationwide presence, but the SIP
> trunking is centralized. They've had enough issues with one DID service
> failing and their customers having to report the issue. Ideally, the SIP
> providers would be able to automatically do "something" when they stop
> receiving options pings, or when a certain sip response is received... but
> it doesn't work that way with the behemoth phone companies.
>
> The way it works now is that MOST issues are able to be caught
> successfully with internal monitoring... but others such certain NPA-NXX
> can't call another NPA-NXX, or carrier interconnects such as "Verizon
> wireless customers cannot call Sprint toll free numbers from area code
> 555"  These odd scenarios are what we are looking to solve. I understand
> this is potentially huge, but I think if we could automate calls to about
> 10 different numbers, that would cover enough of the ingress and carrier
> combinations that it would make a HUGE difference.
>
> I've thought of spinning up an Asterisk and somehow automating the echo
> test feature. I've also thought about using the Twilio API to test if calls
> are successful. Both of these are complicated and prone to issues... so if
> there is a hosted or cloud solution that is already available, please let
> me know.
>
> Any suggestions or more than welcome.
>
> Thanks,
> Nick
> ___
> cisco-voip mailing list
> cisco-voip@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-voip
>
___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

Re: [cisco-voip] Automated PSTN ingress call regression testing?

[Kent Roberts]

I’d have to look but rtmt might be able to report if calls fall below x value   
If so that would be a really quick step while you look at options


Kent

> On Mar 7, 2018, at 12:56, Nick Barnett  wrote:
> 
> A client has a need for an off site solution that will make test calls to 
> their numbers and report when there are issues. I understand that this is 
> very vague, but they are interested in hearing about any and all solutions.
> 
> They have several SIP carriers and a nationwide presence, but the SIP 
> trunking is centralized. They've had enough issues with one DID service 
> failing and their customers having to report the issue. Ideally, the SIP 
> providers would be able to automatically do "something" when they stop 
> receiving options pings, or when a certain sip response is received... but it 
> doesn't work that way with the behemoth phone companies.
> 
> The way it works now is that MOST issues are able to be caught successfully 
> with internal monitoring... but others such certain NPA-NXX can't call 
> another NPA-NXX, or carrier interconnects such as "Verizon wireless customers 
> cannot call Sprint toll free numbers from area code 555"  These odd scenarios 
> are what we are looking to solve. I understand this is potentially huge, but 
> I think if we could automate calls to about 10 different numbers, that would 
> cover enough of the ingress and carrier combinations that it would make a 
> HUGE difference.
> 
> I've thought of spinning up an Asterisk and somehow automating the echo test 
> feature. I've also thought about using the Twilio API to test if calls are 
> successful. Both of these are complicated and prone to issues... so if there 
> is a hosted or cloud solution that is already available, please let me know.
> 
> Any suggestions or more than welcome.
> 
> Thanks,
> Nick
> ___
> cisco-voip mailing list
> cisco-voip@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-voip
___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip

[cisco-voip] Automated PSTN ingress call regression testing?

[Nick Barnett]

A client has a need for an off site solution that will make test calls to
their numbers and report when there are issues. I understand that this is
very vague, but they are interested in hearing about any and all solutions.

They have several SIP carriers and a nationwide presence, but the SIP
trunking is centralized. They've had enough issues with one DID service
failing and their customers having to report the issue. Ideally, the SIP
providers would be able to automatically do "something" when they stop
receiving options pings, or when a certain sip response is received... but
it doesn't work that way with the behemoth phone companies.

The way it works now is that MOST issues are able to be caught successfully
with internal monitoring... but others such certain NPA-NXX can't call
another NPA-NXX, or carrier interconnects such as "Verizon wireless
customers cannot call Sprint toll free numbers from area code 555"  These
odd scenarios are what we are looking to solve. I understand this is
potentially huge, but I think if we could automate calls to about 10
different numbers, that would cover enough of the ingress and carrier
combinations that it would make a HUGE difference.

I've thought of spinning up an Asterisk and somehow automating the echo
test feature. I've also thought about using the Twilio API to test if calls
are successful. Both of these are complicated and prone to issues... so if
there is a hosted or cloud solution that is already available, please let
me know.

Any suggestions or more than welcome.

Thanks,
Nick
___
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip