RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Well the alternative to the 3rd party software is to use Microsoft's SRVANY free utility as the service wrapper and their INSTSRV utility to install the service (see http://support.microsoft.com/kb/137890) and that would not require a registry hack but frankly it's not a reliable mechanism for this kind of thing because it cannot detect when the AX client that SRVANY wraps has died. This means that it can't auto restart a dead client nor alert you to the failure. I would have thought that you would need multiple batch server clients to handle the multiple groups that you need. I suppose it might be practical to start the BatchRun class multiple times inside the same client but that wouldn't result in multiple batch jobs running in parallel. Having said that, if that's what you want to do then a relatively simple change to the SysStartupCmdBatchRun.infoRun() method that assumes that the class instance variable parm is a comma separated list of batch groups and loops through that list starting one BatchRun object for each group in the list would do the job. Malcolm From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of James Flavell Sent: 02 June 2010 17:00 To: development-axapta@yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Thanks Malcolm, Yes using the 3rd party software mentioned earlier rather than the registry hack (because I have had 3 different persons try to get this resgitry hack to work but all failed) Also the final piece in the puzzle is how to modify the startup classes to accept more than one batch group in the parameter (we want to start 3 in each company) but I think the programmer should be able to provide a solution for this without too much issues. But if anyone has already done feel free to share J Thanks again to everyone it has been great to have had your inputs on solutions and experiences and I hope to be able to sow back into the group with some of my own for other people's problems (although I am not really a developer in AX these days) James From: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com] On Behalf Of Malcolm Burtt Sent: 02 June 2010 20:05 To: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? ...and once you have that change in place you can run the batch server as a Windows service. Malcolm From: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com ] On Behalf Of James Flavell Sent: 02 June 2010 10:34 To: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Sorry Mike a quick google gave me a detailed example to try out for this Thanks again James From: James Flavell [mailto:djf1...@gmail.commailto:djf1994%40gmail.com mailto:djf1994%40gmail.com mailto:djf1994%40gmail.com] Sent: 02 June 2010 17:31 To: 'development-axapta@yahoogroups.commailto:%27development-axapta%40yahoogroups.com mailto:%27development-axapta%40yahoogroups.com mailto:%27development-axapta%40yahoogroups.com' Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Thanks Mike Can you just clarify for me about: We have a specific ax client configuration file on the batch server with the batch parameter in the 'command to run at application startup' You mean I can put a parameter in the AX config that will start the batch process running for certain batch groups? (i.e. no need to modify the SysStartup class?) Thank you very much James From: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com] On Behalf Of Mike Sent: 01 June 2010 21:13 To: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James, There are a couple of different ways of handling the restart
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Thanks Malcolm once again Just wondering is there any counter or tool to monitor memory usage in a specific process (i.e. AX32Serve.exe)? Monitoring the total memory or available is going to be a little tough I feel Also just to add is this 2GB limit still in AX2009? And does this mean having 4 GB RAM is all you need for an AOS server (given you can't use more than 2GB and another 2GB for OS should be sufficient)? Thanks James -Original Message- From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of Malcolm Burtt Sent: 30 May 2010 01:56 To: development-axapta@yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James The AX AOS will crash as it approaches 2GB of memory use and its likely that performance would degrade as it nears this upper limit. It could be that you have uncovered a memory leak in AX which might account for this but it could also be that you just have too many users that run memory hungry processes at the same time and simply need another AOS to spread the load. You could use performance monitor to track memory use over time to see if you can get a better handle on whther it is a leak or you could simply grab a new kernel (there are some memory leak fixes in newer kernels as I've been dealing with a leak problem for one of my clients over the last couple of months and the signs are good that the latest kernel has resolved the problem for them. Regards Malcolm Burtt Product Development Manager Touchstone Ltd From: development-axapta@yahoogroups.com [development-axa...@yahoogroups.com] On Behalf Of James Flavell [djf1...@gmail.com] Sent: 25 May 2010 13:18 To: development-axapta@yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] Yahoo! Groups Links
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Hi All, I have run through the same issue in different sites runing Ax 4.0 SP2 and Ax 2009 SP1.I figured about the AOSes will start giving those errors and directly goes down.I made certain monitoring and it was basically because of certainn functions and reports that cause this issues. In my case the Financial statement Report was run with a focus of five dimensions in there ,the report was freezing the client session trying to fetch the report.I was monitoring the memory on the server that user was accessing and it goes down when the unilized memory reaches to 3GB,I am not sure why 3! This case is still logged with MS.and do not know if they found a solution or not. Regards Abdulrahman Absi --- On Mon, 5/31/10, James Flavell djf1...@gmail.com wrote: From: James Flavell djf1...@gmail.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? To: development-axapta@yahoogroups.com Date: Monday, May 31, 2010, 11:33 AM Thanks Martin We upgraded the kernel to solve unstable problems of user sessions crashing or getting message about 'not enough resources so your session will be terminated' and I agreed it has helped a lot as the whole thing was stable for almost 2 months (rather than the usual few days) :) But then this one happened and I guess it was the 2GB limit. They are only 20+ users so we will just have to try to setup a start and stop routine (the main challenge to this is restarting the batch client which I asked about in previous postings in this group) Thanks again James -Original Message- From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of Malcolm Burtt Sent: 30 May 2010 01:56 To: development-axapta@yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James The AX AOS will crash as it approaches 2GB of memory use and its likely that performance would degrade as it nears this upper limit. It could be that you have uncovered a memory leak in AX which might account for this but it could also be that you just have too many users that run memory hungry processes at the same time and simply need another AOS to spread the load. You could use performance monitor to track memory use over time to see if you can get a better handle on whther it is a leak or you could simply grab a new kernel (there are some memory leak fixes in newer kernels as I've been dealing with a leak problem for one of my clients over the last couple of months and the signs are good that the latest kernel has resolved the problem for them. Regards Malcolm Burtt Product Development Manager Touchstone Ltd From: development-axapta@yahoogroups.com [development-axa...@yahoogroups.com] On Behalf Of James Flavell [djf1...@gmail.com] Sent: 25 May 2010 13:18 To: development-axapta@yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] Yahoo! Groups Links [Non-text portions of this message have been removed]
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Cheers Absi for sharing that very useful info J Apart from Financial Statement was there any other reports or functions that should signs of causing this? Thanks James From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of abd absi Sent: 31 May 2010 22:25 To: development-axapta@yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi All, I have run through the same issue in different sites runing Ax 4.0 SP2 and Ax 2009 SP1.I figured about the AOSes will start giving those errors and directly goes down.I made certain monitoring and it was basically because of certainn functions and reports that cause this issues. In my case the Financial statement Report was run with a focus of five dimensions in there ,the report was freezing the client session trying to fetch the report.I was monitoring the memory on the server that user was accessing and it goes down when the unilized memory reaches to 3GB,I am not sure why 3! This case is still logged with MS.and do not know if they found a solution or not. Regards Abdulrahman Absi --- On Mon, 5/31/10, James Flavell djf1...@gmail.com mailto:djf1994%40gmail.com wrote: From: James Flavell djf1...@gmail.com mailto:djf1994%40gmail.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Date: Monday, May 31, 2010, 11:33 AM Thanks Martin We upgraded the kernel to solve unstable problems of user sessions crashing or getting message about 'not enough resources so your session will be terminated' and I agreed it has helped a lot as the whole thing was stable for almost 2 months (rather than the usual few days) :) But then this one happened and I guess it was the 2GB limit. They are only 20+ users so we will just have to try to setup a start and stop routine (the main challenge to this is restarting the batch client which I asked about in previous postings in this group) Thanks again James -Original Message- From: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com ] On Behalf Of Malcolm Burtt Sent: 30 May 2010 01:56 To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James The AX AOS will crash as it approaches 2GB of memory use and its likely that performance would degrade as it nears this upper limit. It could be that you have uncovered a memory leak in AX which might account for this but it could also be that you just have too many users that run memory hungry processes at the same time and simply need another AOS to spread the load. You could use performance monitor to track memory use over time to see if you can get a better handle on whther it is a leak or you could simply grab a new kernel (there are some memory leak fixes in newer kernels as I've been dealing with a leak problem for one of my clients over the last couple of months and the signs are good that the latest kernel has resolved the problem for them. Regards Malcolm Burtt Product Development Manager Touchstone Ltd From: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com [development-axa...@yahoogroups.com mailto:development-axapta%40yahoogroups.com ] On Behalf Of James Flavell [djf1...@gmail.com mailto:djf1994%40gmail.com ] Sent: 25 May 2010 13:18 To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] Yahoo! Groups Links [Non-text portions of this message have been removed] [Non-text portions of this message have been removed]
Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Hi James, There are a couple of different ways of handling the restart of the batch server. The best way is to get an extra batch server license and have a AOS service running on the batch server and have the batch client connect to it's on AOS service. This way users can't mess with the batch AOS and the batch AOS can't mess with the users. Before we had the separate batch AOS service, we ran one script on the batch server to reboot the batch server at the same time as the other script was stopping the AOS service on the AOS server. We have a specific ax client configuration file on the batch server with the batch parameter in the 'command to run at application startup' Then as a startup task we have a script that starts the ax client: Run C:\Program Files\Microsoft Dynamics AX\40_sp2\Client\AX40_Batch.axc From: James Flavell Sent: Monday, May 31, 2010 12:59 AM To: development-axapta@yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Thanks Mike J I don't believe we have any 3rd party apps that connect via business connector so it must be one of the normal clients L The only external party I can think of is the SQL OLAP being accessed via the AX client Pivot View analysis (or the pushing of data to the SQL OLAP during the cube processing) but I don't think this use business connector right? For stopping and starting since this is AX4.0 do you have any suggestions for getting the batch client running again? I believe I have to do something about the startup class or the like and then have a batch file that starts a client with some startup parameter or something right? Thanks again for your great help, much appreciated J James From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of Mike Sent: 28 May 2010 20:52 To: development-axapta@yahoogroups.com Subject: Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? I would focus on whatever 3rd party apps you have on that AOS that connect via the business connector. That's going to be the source of an RPC error on the AOS. Yes, the AOS service crashes when it runs out of memory and it runs slow as it approaches the upper limits. You should restart the aos service periodically to prevent this from happening. pick a time of day when it affects the least amount of people. you could even create 2 .bat files with 'net stop AOS$03' and 'net start AOS$03' and schedule them to run 2 minutes apart at a time when no one is on the system. From: James Flavell Sent: Tuesday, May 25, 2010 8:18 AM To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] [Non-text portions of this message have been removed] [Non-text portions of this message have been removed] [Non-text portions of this message have been removed]
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Thanks Mike Can you just clarify for me about: We have a specific ax client configuration file on the batch server with the batch parameter in the 'command to run at application startup' You mean I can put a parameter in the AX config that will start the batch process running for certain batch groups? (i.e. no need to modify the SysStartup class?) Thank you very much James From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of Mike Sent: 01 June 2010 21:13 To: development-axapta@yahoogroups.com Subject: Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James, There are a couple of different ways of handling the restart of the batch server. The best way is to get an extra batch server license and have a AOS service running on the batch server and have the batch client connect to it's on AOS service. This way users can't mess with the batch AOS and the batch AOS can't mess with the users. Before we had the separate batch AOS service, we ran one script on the batch server to reboot the batch server at the same time as the other script was stopping the AOS service on the AOS server. We have a specific ax client configuration file on the batch server with the batch parameter in the 'command to run at application startup' Then as a startup task we have a script that starts the ax client: Run C:\Program Files\Microsoft Dynamics AX\40_sp2\Client\AX40_Batch.axc From: James Flavell Sent: Monday, May 31, 2010 12:59 AM To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Thanks Mike J I don't believe we have any 3rd party apps that connect via business connector so it must be one of the normal clients L The only external party I can think of is the SQL OLAP being accessed via the AX client Pivot View analysis (or the pushing of data to the SQL OLAP during the cube processing) but I don't think this use business connector right? For stopping and starting since this is AX4.0 do you have any suggestions for getting the batch client running again? I believe I have to do something about the startup class or the like and then have a batch file that starts a client with some startup parameter or something right? Thanks again for your great help, much appreciated J James From: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com ] On Behalf Of Mike Sent: 28 May 2010 20:52 To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? I would focus on whatever 3rd party apps you have on that AOS that connect via the business connector. That's going to be the source of an RPC error on the AOS. Yes, the AOS service crashes when it runs out of memory and it runs slow as it approaches the upper limits. You should restart the aos service periodically to prevent this from happening. pick a time of day when it affects the least amount of people. you could even create 2 .bat files with 'net stop AOS$03' and 'net start AOS$03' and schedule them to run 2 minutes apart at a time when no one is on the system. From: James Flavell Sent: Tuesday, May 25, 2010 8:18 AM To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] [Non-text portions of this message have been removed] [Non-text portions of this message have been removed] [Non-text portions of this message have been removed] [Non-text portions of this message have been removed]
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Hi James Forget that, I was having a moment of madness about the ax32.exe processes. :) Malcolm From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of James Flavell Sent: 02 June 2010 10:12 To: development-axapta@yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? As always thank you very much Malcolm for sharing your insights J Will try to get this counter setup J I was not aware that were other AX32.exe processes running on the AOS server (I thought these were client side only) . I will go take a snoop around to find these when I have sometime Thanks again James From: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com] On Behalf Of Malcolm Burtt Sent: 02 June 2010 16:39 To: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James You can monitor the Private Bytes perfmon Process counter for the Ax32Serv.exe process. If that starts to approach 2GB then you know you're heading for a crash. I'm not sure that you can be that limited with your memory. Sure the AOS process itself can only address 2GB but then there are also Ax32.exe processes for each connected client and they'll need memory of their own and, of course, the OS is consuming some memory. Malcolm From: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com ] On Behalf Of James Flavell Sent: 01 June 2010 08:04 To: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Thanks Malcolm once again Just wondering is there any counter or tool to monitor memory usage in a specific process (i.e. AX32Serve.exe)? Monitoring the total memory or available is going to be a little tough I feel Also just to add is this 2GB limit still in AX2009? And does this mean having 4 GB RAM is all you need for an AOS server (given you can't use more than 2GB and another 2GB for OS should be sufficient)? Thanks James -Original Message- From: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com] On Behalf Of Malcolm Burtt Sent: 30 May 2010 01:56 To: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James The AX AOS will crash as it approaches 2GB of memory use and its likely that performance would degrade as it nears this upper limit. It could be that you have uncovered a memory leak in AX which might account for this but it could also be that you just have too many users that run memory hungry processes at the same time and simply need another AOS to spread the load. You could use performance monitor to track memory use over time to see if you can get a better handle on whther it is a leak or you could simply grab a new kernel (there are some memory leak fixes in newer kernels as I've been dealing with a leak problem for one of my clients over the last couple of months and the signs are good that the latest kernel has resolved the problem for them. Regards Malcolm Burtt Product Development Manager Touchstone Ltd From: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com [development-axa...@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com] On Behalf Of James Flavell [djf1...@gmail.commailto:djf1994%40gmail.com mailto:djf1994%40gmail.com mailto:djf1994%40gmail.com] Sent: 25 May 2010 13:18 To: development-axapta@yahoogroups.commailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Sorry Mike a quick google gave me a detailed example to try out for this Thanks again James From: James Flavell [mailto:djf1...@gmail.com] Sent: 02 June 2010 17:31 To: 'development-axapta@yahoogroups.com' Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Thanks Mike Can you just clarify for me about: We have a specific ax client configuration file on the batch server with the batch parameter in the 'command to run at application startup' You mean I can put a parameter in the AX config that will start the batch process running for certain batch groups? (i.e. no need to modify the SysStartup class?) Thank you very much James From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of Mike Sent: 01 June 2010 21:13 To: development-axapta@yahoogroups.com Subject: Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James, There are a couple of different ways of handling the restart of the batch server. The best way is to get an extra batch server license and have a AOS service running on the batch server and have the batch client connect to it's on AOS service. This way users can't mess with the batch AOS and the batch AOS can't mess with the users. Before we had the separate batch AOS service, we ran one script on the batch server to reboot the batch server at the same time as the other script was stopping the AOS service on the AOS server. We have a specific ax client configuration file on the batch server with the batch parameter in the 'command to run at application startup' Then as a startup task we have a script that starts the ax client: Run C:\Program Files\Microsoft Dynamics AX\40_sp2\Client\AX40_Batch.axc From: James Flavell Sent: Monday, May 31, 2010 12:59 AM To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Thanks Mike J I don't believe we have any 3rd party apps that connect via business connector so it must be one of the normal clients L The only external party I can think of is the SQL OLAP being accessed via the AX client Pivot View analysis (or the pushing of data to the SQL OLAP during the cube processing) but I don't think this use business connector right? For stopping and starting since this is AX4.0 do you have any suggestions for getting the batch client running again? I believe I have to do something about the startup class or the like and then have a batch file that starts a client with some startup parameter or something right? Thanks again for your great help, much appreciated J James From: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com [mailto:development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com ] On Behalf Of Mike Sent: 28 May 2010 20:52 To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? I would focus on whatever 3rd party apps you have on that AOS that connect via the business connector. That's going to be the source of an RPC error on the AOS. Yes, the AOS service crashes when it runs out of memory and it runs slow as it approaches the upper limits. You should restart the aos service periodically to prevent this from happening. pick a time of day when it affects the least amount of people. you could even create 2 .bat files with 'net stop AOS$03' and 'net start AOS$03' and schedule them to run 2 minutes apart at a time when no one is on the system. From: James Flavell Sent: Tuesday, May 25, 2010 8:18 AM To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] [Non-text portions of this message have been removed] [Non-text
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Thanks Mike J I don't believe we have any 3rd party apps that connect via business connector so it must be one of the normal clients L The only external party I can think of is the SQL OLAP being accessed via the AX client Pivot View analysis (or the pushing of data to the SQL OLAP during the cube processing) but I don't think this use business connector right? For stopping and starting since this is AX4.0 do you have any suggestions for getting the batch client running again? I believe I have to do something about the startup class or the like and then have a batch file that starts a client with some startup parameter or something right? Thanks again for your great help, much appreciated J James From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of Mike Sent: 28 May 2010 20:52 To: development-axapta@yahoogroups.com Subject: Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? I would focus on whatever 3rd party apps you have on that AOS that connect via the business connector. That's going to be the source of an RPC error on the AOS. Yes, the AOS service crashes when it runs out of memory and it runs slow as it approaches the upper limits. You should restart the aos service periodically to prevent this from happening. pick a time of day when it affects the least amount of people. you could even create 2 .bat files with 'net stop AOS$03' and 'net start AOS$03' and schedule them to run 2 minutes apart at a time when no one is on the system. From: James Flavell Sent: Tuesday, May 25, 2010 8:18 AM To: development-axapta@yahoogroups.com mailto:development-axapta%40yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] [Non-text portions of this message have been removed] [Non-text portions of this message have been removed]
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Hi James The AX AOS will crash as it approaches 2GB of memory use and its likely that performance would degrade as it nears this upper limit. It could be that you have uncovered a memory leak in AX which might account for this but it could also be that you just have too many users that run memory hungry processes at the same time and simply need another AOS to spread the load. You could use performance monitor to track memory use over time to see if you can get a better handle on whther it is a leak or you could simply grab a new kernel (there are some memory leak fixes in newer kernels as I've been dealing with a leak problem for one of my clients over the last couple of months and the signs are good that the latest kernel has resolved the problem for them. Regards Malcolm Burtt Product Development Manager Touchstone Ltd From: development-axapta@yahoogroups.com [development-axa...@yahoogroups.com] On Behalf Of James Flavell [djf1...@gmail.com] Sent: 25 May 2010 13:18 To: development-axapta@yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] Yahoo! Groups Links * To visit your group on the web, go to: http://groups.yahoo.com/group/development-axapta/ * Your email settings: Individual Email | Traditional * To change settings online go to: http://groups.yahoo.com/group/development-axapta/join (Yahoo! ID required) * To change settings via email: development-axapta-dig...@yahoogroups.com development-axapta-fullfeatu...@yahoogroups.com * To unsubscribe from this group, send an email to: development-axapta-unsubscr...@yahoogroups.com * Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
Thanks Martin We upgraded the kernel to solve unstable problems of user sessions crashing or getting message about 'not enough resources so your session will be terminated' and I agreed it has helped a lot as the whole thing was stable for almost 2 months (rather than the usual few days) :) But then this one happened and I guess it was the 2GB limit. They are only 20+ users so we will just have to try to setup a start and stop routine (the main challenge to this is restarting the batch client which I asked about in previous postings in this group) Thanks again James -Original Message- From: development-axapta@yahoogroups.com [mailto:development-axa...@yahoogroups.com] On Behalf Of Malcolm Burtt Sent: 30 May 2010 01:56 To: development-axapta@yahoogroups.com Subject: RE: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi James The AX AOS will crash as it approaches 2GB of memory use and its likely that performance would degrade as it nears this upper limit. It could be that you have uncovered a memory leak in AX which might account for this but it could also be that you just have too many users that run memory hungry processes at the same time and simply need another AOS to spread the load. You could use performance monitor to track memory use over time to see if you can get a better handle on whther it is a leak or you could simply grab a new kernel (there are some memory leak fixes in newer kernels as I've been dealing with a leak problem for one of my clients over the last couple of months and the signs are good that the latest kernel has resolved the problem for them. Regards Malcolm Burtt Product Development Manager Touchstone Ltd From: development-axapta@yahoogroups.com [development-axa...@yahoogroups.com] On Behalf Of James Flavell [djf1...@gmail.com] Sent: 25 May 2010 13:18 To: development-axapta@yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] Yahoo! Groups Links
Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
There is also the windows2003 RPC bug fix that you could try. The only time I have ever heard it working for ax problems involved errors when users printed to pdf. http://support.microsoft.com/kb/913184 An RPC client cannot connect to an RPC server that is running Windows Server 2003 SP1, and event ID 180 is logged From: James Flavell Sent: Tuesday, May 25, 2010 8:18 AM To: development-axapta@yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] [Non-text portions of this message have been removed]
Re: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas?
I would focus on whatever 3rd party apps you have on that AOS that connect via the business connector. That's going to be the source of an RPC error on the AOS. Yes, the AOS service crashes when it runs out of memory and it runs slow as it approaches the upper limits. You should restart the aos service periodically to prevent this from happening. pick a time of day when it affects the least amount of people. you could even create 2 .bat files with 'net stop AOS$03' and 'net start AOS$03' and schedule them to run 2 minutes apart at a time when no one is on the system. From: James Flavell Sent: Tuesday, May 25, 2010 8:18 AM To: development-axapta@yahoogroups.com Subject: [development-axapta] Live AOS failure RPC exception 14 reported in event log many many times per second - any ideas? Hi everyone We have an AX4.0 SP2 with kernel 4.0.2503.953 Today the user reported very slow performance and the AOS crashed later on When we looked in the event viewer there was an error: Object Server 01: RPC error: RPC exception 14 occurred in session 3 This was logged maybe 50 to 100 times in a single second every second!!! I searched but could find no mention of RCP 14 error anywhere (including partner source) Does anyone have any idea what can cause this error or similar kind of mass logging in the event viewer every second? Also I heard the Ax4.0 AOS is likely to crash when the Ax32Serv.exe process reaches around the 2GB limit. Has anyone heard about this? Thanks James [Non-text portions of this message have been removed] [Non-text portions of this message have been removed]
