The drop table statements show threads shutting down and cleaning up temporary tables. The number at the end of the name is the thread ID so I suspect this is the preload threads finishing work rather than a thread crashing – you’re not seeing any stack traces in the arerror.log re you?
It looks a lot like some sort of metadata corruption so you may be able to get a hint of what the issue is with the –t startup option. Copy the arserver.exe line from armonitor.cfg and run it at the command line but insert –t after the arserver.exe – this will create a arstartup_PID.log and cause some validation of the metadata to be run and logged. Once you have that you’re likely to need a support case with BMC to get some further insight as to where it is getting stuck. Mark From: Action Request System discussion list(ARSList) [mailto:[email protected]] On Behalf Of BradRemedy Sent: 20 January 2015 06:23 To: [email protected] Subject: Re: Remedy 8.1 SP2 - Hanging on Startup ** Hi Jason I did get SQL profiler running and checked what was being called by remedy and found the following entries: SQL:BatchStarting DROP TABLE AR0int5212 182 2015-01-20 07:38:58.553 SQL:BatchStarting DROP TABLE AR0int8000 166 2015-01-20 07:38:58.553 SQL:BatchStarting DROP TABLE AR0int7664 178 2015-01-20 07:38:58.560 SQL:BatchStarting DROP TABLE AR0float5212 182 2015-01-20 07:38:58.560 SQL:BatchStarting DROP TABLE AR0float8000 166 2015-01-20 07:38:58.560 SQL:BatchStarting DROP TABLE AR0float7664 178 2015-01-20 07:38:58.560 SQL:BatchStarting DROP TABLE AR0char5212 182 2015-01-20 07:38:58.560 SQL:BatchStarting DROP TABLE AR0char8000 166 2015-01-20 07:38:58.560 SQL:BatchStarting DROP TABLE AR0char7664 178 2015-01-20 07:38:58.560 SQL:BatchStarting DROP TABLE AR0decimal5212 182 2015-01-20 07:38:58.560 SQL:BatchStarting DROP TABLE AR0decimal8000 166 2015-01-20 07:38:58.560 Audit Logout 182 2015-01-20 07:38:58.180 SQL:BatchStarting DROP TABLE AR0decimal7664 178 2015-01-20 07:38:58.560 Audit Logout 166 2015-01-20 07:38:57.217 Audit Logout 178 2015-01-20 07:38:58.020 SQL:BatchStarting DROP TABLE AR0int7732 171 2015-01-20 07:38:58.563 SQL:BatchStarting DROP TABLE AR0float7732 171 2015-01-20 07:38:58.567 SQL:BatchStarting DROP TABLE AR0char7732 171 2015-01-20 07:38:58.570 SQL:BatchStarting DROP TABLE AR0decimal7732 171 2015-01-20 07:38:58.570 Audit Logout 171 2015-01-20 07:38:57.280 SQL:BatchStarting DROP TABLE AR0int6804 183 2015-01-20 07:38:58.627 SQL:BatchStarting DROP TABLE AR0float6804 183 2015-01-20 07:38:58.627 SQL:BatchStarting DROP TABLE AR0char6804 183 2015-01-20 07:38:58.627 SQL:BatchStarting DROP TABLE AR0decimal6804 183 2015-01-20 07:38:58.633 Not sure what the system is trying to do - should i leave it and see ? I have also logged this with BMC Software to see if they have any ideas. As always, thanks for the suggestions and help guys - it is really appreciated. I am hoping my company approves my proposal to attend the BMC Engage event this year so that I can meet you guys and thank you in person. Cheers Brad On Tue, Jan 20, 2015 at 6:36 AM, Jason Miller <[email protected]<mailto:[email protected]>> wrote: ** I would definitely run a trace on the db as LJ mentioned to see what if any activity is going on. You can also turn on a startup log by adding a switch in you armonitor.cfg/config file. I can't think of it off the top of my head but it has been mentioned on the list before and should be on docs.bmc.com<http://docs.bmc.com>. Wait... I think it might be -i. It gets added to the line that actually starts arserver. Jason On Jan 19, 2015 8:28 PM, "BradRemedy" <[email protected]<mailto:[email protected]>> wrote: ** Hi, I checked in the ar.cfg file and the "Record-Object-Relationship" is set to "F". As a test I restarted the entire server yet the service still says "Starting" with the memory usage for the arserver.exe process sitting at 390mb. It will stay like this for hours and wont start. There is nothing in the arerror log file besides the remedy version information and the startup log file has the same entries as per my original post. The last line is still "Mon Jan 19 2015 13:40:05.7000 <Startup> <TID: 0000004696> Loading field mapping information" Thanks in Advance Brad On Mon, Jan 19, 2015 at 3:41 PM, Harshad Wagh <[email protected]<mailto:[email protected]>> wrote: ** Hi Brad, I wonder if there is "Record Object Relationship" is set to true in under "Configuration" tab of "Server Information" tab then Remedy Application Service takes longer time to start. you can check this in ar.cfg/ar.conf file and set it to "F" and try to start the Remedy application service. Thanks Harshad -------- Original Message -------- Subject: Remedy 8.1 SP2 - Hanging on Startup From: BradRemedy <[email protected]<mailto:[email protected]>> Date: Mon, January 19, 2015 12:30 am To: [email protected]<mailto:[email protected]> ** Hi Guys We have a remedy 8.1 installation with dy The arstartup log file has the following entries: Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Server version = 8.1.02 201408260235 Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Enabling Low Fragmentation Heap succeeded Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> InstallDir = C:\Program Files\BMC Software\ARSystem Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Set up thread control block with key = 0 Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Initialize thread local storage block Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Initialize multiple-byte environment Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> LocalLanguage = ENS Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> ViewDir = en Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Initialize License library Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> LicenseFilename = C:\Program Files\Common Files\AR System\Licenses\RemedyAppServer\arsystem.lic Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Initialize Language setting and locale Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Initialize the Decimal Math library Mon Jan 19 2015 08:54:39.8920 <Startup> <TID: 0000007260> Open shared catalog Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Create Mutexes Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Load encryption shared library Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Load encryption static functions Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize default configuration information Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize parse environment Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize date time information Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize ApiGuidMap Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize notification strings Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize RPC queue type strings Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize filter strings Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize escalation strings Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Load system configuration file Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> arsCodeSet=windows-1252[0] arDbCodeSet=windows-1252[0] Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize pending lists Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Create cache read write lock Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Create full text status read write lock Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Create modified date field Id List lock Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize fork proxy Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize user cache Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Open log files Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Initialize XML parser Mon Jan 19 2015 08:54:39.9080 <Startup> <TID: 0000007260> Load AR Extensions Mon Jan 19 2015 08:54:39.9390 <Startup> <TID: 0000007260> Initialize process communication mechanism Mon Jan 19 2015 08:54:39.9390 <Startup> <TID: 0000007260> Initialize thread list Mon Jan 19 2015 08:54:39.9390 <Startup> <TID: 0000007260> Check multiple servers Mon Jan 19 2015 08:54:39.9390 <Startup> <TID: 0000007260> Initialize dead thread list Mon Jan 19 2015 08:54:39.9390 <Startup> <TID: 0000007260> Initialize timed calls Mon Jan 19 2015 08:54:39.9390 <Startup> <TID: 0000007260> CreateRPCQueue(min=1, max=1, rpc=390600) Mon Jan 19 2015 08:54:39.9390 <Startup> <TID: 0000007260> Await cache ready ... Mon Jan 19 2015 08:54:39.9550 <Startup> <TID: 0000001020> Check licensing Mon Jan 19 2015 08:54:40.0010 <Startup> <TID: 0000001020> InitializeServerCache: PerformPreCachePatchDatabaseUpgrades End Mon Jan 19 2015 08:54:40.0010 <Startup> <TID: 0000001020> InitializeServerCache: LoadInitialSchemaInfo Begin Mon Jan 19 2015 08:54:40.0010 <Startup> <TID: 0000001020> Loading group list Mon Jan 19 2015 08:54:40.0330 <Startup> <TID: 0000001020> Initialize license pools Mon Jan 19 2015 08:54:40.0330 <Startup> <TID: 0000001020> Loading schema properties Mon Jan 19 2015 08:54:40.2670 <Startup> <TID: 0000001020> Starting preload threads Mon Jan 19 2015 08:54:40.2820 <Startup> <TID: 0000001020> Loading list of fields for all schemas Mon Jan 19 2015 08:54:44.2130 <Startup> <TID: 0000001020> Loading schema/group information Mon Jan 19 2015 08:54:44.4470 <Startup> <TID: 0000001020> Loading schema getlist/index/archive/audit information Mon Jan 19 2015 08:54:45.1490 <Startup> <TID: 0000001020> Loading schema/vui information Mon Jan 19 2015 08:54:45.3370 <Startup> <TID: 0000001020> Loading field mapping information ************************************************************************* Anyone had this before? Appears that the field mapping information is corrupt or running into a problem and I am not sure how best to move forward with this. Any ideas would be appreciated Cheers Brad _ARSlist: "Where the Answers Are" and have been for 20 years_ _ARSlist: "Where the Answers Are" and have been for 20 years_ _ARSlist: "Where the Answers Are" and have been for 20 years_ _ARSlist: "Where the Answers Are" and have been for 20 years_ _ARSlist: "Where the Answers Are" and have been for 20 years_ _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org "Where the Answers Are, and have been for 20 years"
