Re: [asterisk-users] Testing Framework
Matt Riddell [EMAIL PROTECTED] writes: Hi, I propose we start with a list of things that we think should be tested in Asterisk, and means to test them. Any takers? Add to the list? If there is something you believe is mission critical to your business, write up a test case for it, and we'll all try to code something that can run automatically to test it. I think a good way to get new items on the testing list would be for Digium/community to take a look at the bug tracker for released versions and ask 'Was there a way we could have caught this bug?'. Then add that test case to the list. In my mind I would rather have too many tests than not enough! :) -- Kyle Sexton ___ Sign up now for AstriCon 2007! September 25-28th. http://www.astricon.net/ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Testing Framework
I think the testing frame work includes both the components and system testing. I wish to add some more test even though all giants may aware, since i wish to do some contribution to asterisk what ever i can. i am plannig for the framework and addon as given below, expecting techies advise in this, Testing frame work may contain testing internally and externally, i mean to say internally, some client may stick into the problem of voice quality, when more phones on the PBX, that time, we may not leave the system from the network, their call may get interrupted, that time internal testing daemon will work for the performance analysis till the lelvel of without affecting present calls. externally means PBX system on production for performance analysis, as per the test case started in the mail. internal and external testing may be configurable at the run time, through some verbose like variable configurable at run time, addon should have the capability of releasing the testing call bandwidth, whenever PBX gets new call, this might be simple example. procesding with Testing frame work requirements, call on volume, 1. SIP - SIP, multiple SIP clients support, through which we can either direct the call for testing to another client registered in testing frame work, or return back to the different client registered in the same testing frame work, when the call incoming and outgoing call are handled in single framework point, wave analysis also cane be done with the script and performance can be easily evaluated. On production performance testing, by connecting multiple testing framework point to the PBX, having the sending files in all the frame work, analysis and performance evaluation can be done very easily, i also think that once it is done for SIP with compatiblity of like channel driver, we can adapt IAX2, anything we want. i think this type of testing would make the system stable and provide good support on system on running also. Hariharan.V. RD Engineer, NEEVEE Technologies, On 9/3/07, dave cantera [EMAIL PROTECTED] wrote: matt, are you looking for unit testing of the * components or systems testing, testing the finished product? or both? I think you are onto something here... I hope it takes root. I would say put it in the addons. it would be Great if digium takes it up. it is a smart move for them to foster, cajole, nudge, and support it. call volume I would leave to others as different processors, O/S, builds, kernel versions, and configurations will have too many variables. I was playing with the idea of monitoring multiple * systems. perhaps we can start out with testing the components and then migrate the project (future) to one pbx monitor the other. we will need scripts to initiate some action, config to make some measurements, the scripts to gather the results into a nice neat little summary report. you will want to take the human aspect out of the picture as much as possible. for example: on pbx A * create a recording in multiple formats .gsm, .wav, etc. * initiate a script to generate 5,10, or 25 calls to pbx B and play the file on pbx B * pbx B gets the calls, records them, * copy the recordings from pbx A to pbx B (or have that already done) * have a wave analyzer compare the recordings to the original files (you know I won't be writing that program! :) * report on anomalies *call * *Technology * *recording delta * 1 Zap Provider 1 2% 2 VoIP Provider 2 5% 3 VoIP Provider 2 15% ... VoIP Provider 3 ... let me know what you think! daveC Matt Riddell wrote: Hash: SHA1 Hi, So, now that we've all complained about the state of testing of Open Source versions of Asterisk, lets do something about it. I propose we start with a list of things that we think should be tested in Asterisk, and means to test them. Maybe we could run certain tests based on the changes between minor versions? Anyway lets start. Call Volumes 1) Call volume up to x channels from SIP to SIP (i.e. sipp) 2) Call volume up to x channels from IAX2 to SIP 3) Call volume up to x channels from IAX2 to IAX2 Application testing 4) Connect x calls between techs to Meetme (leave running for 1 hour) 5) Connect x concurrent calls to VoiceMail Call Centre Testing 6) Send x calls to a queue with no agents in it, leave them holding for x minutes 7) Run x calls against AMD connected to recorded known good files Recording 8) Run x calls recording simultaneously from an automatically generated call, play ulaw/alaw - compare outputs. You get the idea. If people can add to this list, I can start making a few scripts and programs that will test them (as I'm sure others can). If we end up with a complete list, I'm sure some of our
Re: [asterisk-users] Testing Framework
matt, are you looking for unit testing of the * components or systems testing, testing the finished product? or both? I think you are onto something here... I hope it takes root. I would say put it in the addons. it would be Great if digium takes it up. it is a smart move for them to foster, cajole, nudge, and support it. call volume I would leave to others as different processors, O/S, builds, kernel versions, and configurations will have too many variables. I was playing with the idea of monitoring multiple * systems. perhaps we can start out with testing the components and then migrate the project (future) to one pbx monitor the other. we will need scripts to initiate some action, config to make some measurements, the scripts to gather the results into a nice neat little summary report. you will want to take the human aspect out of the picture as much as possible. for example: on pbx A * create a recording in multiple formats .gsm, .wav, etc. * initiate a script to generate 5,10, or 25 calls to pbx B and play the file on pbx B * pbx B gets the calls, records them, * copy the recordings from pbx A to pbx B (or have that already done) * have a wave analyzer compare the recordings to the original files (you know I won't be writing that program! :) * report on anomalies *call * *Technology * *recording delta * 1 Zap Provider 1 2% 2 VoIP Provider 2 5% 3 VoIP Provider 2 15% ... VoIP Provider 3 ... let me know what you think! daveC Matt Riddell wrote: Hash: SHA1 Hi, So, now that we've all complained about the state of testing of Open Source versions of Asterisk, lets do something about it. I propose we start with a list of things that we think should be tested in Asterisk, and means to test them. Maybe we could run certain tests based on the changes between minor versions? Anyway lets start. Call Volumes 1) Call volume up to x channels from SIP to SIP (i.e. sipp) 2) Call volume up to x channels from IAX2 to SIP 3) Call volume up to x channels from IAX2 to IAX2 Application testing 4) Connect x calls between techs to Meetme (leave running for 1 hour) 5) Connect x concurrent calls to VoiceMail Call Centre Testing 6) Send x calls to a queue with no agents in it, leave them holding for x minutes 7) Run x calls against AMD connected to recorded known good files Recording 8) Run x calls recording simultaneously from an automatically generated call, play ulaw/alaw - compare outputs. You get the idea. If people can add to this list, I can start making a few scripts and programs that will test them (as I'm sure others can). If we end up with a complete list, I'm sure some of our individual QA departments can take the responsibility for certain items. The call volume ones are obviously going to either need a live person to dial in at volume and check everything is ok, or a recording which can later be checked. I'm of the opinion that the majority of tests should test individual components, but that we should also form some Application Type frameworks so that we can test integration between Asterisk apps. Any takers? Add to the list? If there is something you believe is mission critical to your business, write up a test case for it, and we'll all try to code something that can run automatically to test it. If we try and keep to ANSI C for the testing apps, Digium should be able to run them on their multi platform machines as well. Should these tests be added to Asterisk-Addons or maintained outside of the tree? Anyway, what do you think? Feasible? I already have a few tests here and I'm sure others have a few too. Lets put them all together and get a framework going. - -- Kind Regards, Matt Riddell Director ___ http://www.venturevoip.com (Great new VoIP end to end solution) http://www.venturevoip.com/news.php (Daily Asterisk News - html) http://feeds.venturevoip.com/AsteriskNews (Daily Asterisk News - rss) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG1yKhDQNt8rg0Kp4RAv5UAJ48tW28T5lWCQIPTwVimyvlhEPJowCgpnE6 OF3L2M/6Hc+YBNL1NFx6dzA= =OXNn -END PGP SIGNATURE- ___ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users -- My wife's sister is in California. I should buy her a Videophone2008! Truly, The Next Best Thing to Being There! -- WorldWideVideoPhones.com 856.380.0894 ___ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing
Re: [asterisk-users] Testing Framework
On 8/30/07, Russell Bryant [EMAIL PROTECTED] wrote: Matt Riddell wrote: Should these tests be added to Asterisk-Addons or maintained outside of the tree? If people start writing test utilities, I would be happy to host them in a subversion repository. Depending on the size of this stuff, it could probably go into the main Asterisk repository. We'll see where things go ... Our company have a scriptable testing utility (it's quite far from being simple to use), but we could share it as basis for testing framework or at least - as example for inspiration. I just talked to management, and this week it will be discussed. I'll write you in case of success. Regards, Atis -- Atis Lezdins, IT Responsible of BEST Riga, [EMAIL PROTECTED] ICQ: 142239285 Skype: atis.lezdins Cell Phone: +371 28806004 [Tele2, Latvia] Work phone: +1 800 7502835 [Toll free, USA] ?BEST? - www.BEST.eu.org ___ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
[asterisk-users] Testing Framework
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, So, now that we've all complained about the state of testing of Open Source versions of Asterisk, lets do something about it. I propose we start with a list of things that we think should be tested in Asterisk, and means to test them. Maybe we could run certain tests based on the changes between minor versions? Anyway lets start. Call Volumes 1) Call volume up to x channels from SIP to SIP (i.e. sipp) 2) Call volume up to x channels from IAX2 to SIP 3) Call volume up to x channels from IAX2 to IAX2 Application testing 4) Connect x calls between techs to Meetme (leave running for 1 hour) 5) Connect x concurrent calls to VoiceMail Call Centre Testing 6) Send x calls to a queue with no agents in it, leave them holding for x minutes 7) Run x calls against AMD connected to recorded known good files Recording 8) Run x calls recording simultaneously from an automatically generated call, play ulaw/alaw - compare outputs. You get the idea. If people can add to this list, I can start making a few scripts and programs that will test them (as I'm sure others can). If we end up with a complete list, I'm sure some of our individual QA departments can take the responsibility for certain items. The call volume ones are obviously going to either need a live person to dial in at volume and check everything is ok, or a recording which can later be checked. I'm of the opinion that the majority of tests should test individual components, but that we should also form some Application Type frameworks so that we can test integration between Asterisk apps. Any takers? Add to the list? If there is something you believe is mission critical to your business, write up a test case for it, and we'll all try to code something that can run automatically to test it. If we try and keep to ANSI C for the testing apps, Digium should be able to run them on their multi platform machines as well. Should these tests be added to Asterisk-Addons or maintained outside of the tree? Anyway, what do you think? Feasible? I already have a few tests here and I'm sure others have a few too. Lets put them all together and get a framework going. - -- Kind Regards, Matt Riddell Director ___ http://www.venturevoip.com (Great new VoIP end to end solution) http://www.venturevoip.com/news.php (Daily Asterisk News - html) http://feeds.venturevoip.com/AsteriskNews (Daily Asterisk News - rss) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG1yKhDQNt8rg0Kp4RAv5UAJ48tW28T5lWCQIPTwVimyvlhEPJowCgpnE6 OF3L2M/6Hc+YBNL1NFx6dzA= =OXNn -END PGP SIGNATURE- ___ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Testing Framework
Matt Riddell wrote: Should these tests be added to Asterisk-Addons or maintained outside of the tree? If people start writing test utilities, I would be happy to host them in a subversion repository. Depending on the size of this stuff, it could probably go into the main Asterisk repository. We'll see where things go ... -- Russell Bryant Software Engineer Digium, Inc. ___ --Bandwidth and Colocation Provided by http://www.api-digital.com-- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
