Abhijeet,
if ARSystemInstalledConfiguration.xml so crucial for the installation why
wouldn’t the pre-check routine tell me that but continues with the installation
instead?
Where in the BMC documentation do I read that the
ARSystemInstalledConfiguration.xml needs to be there and what it should contain
for a successful upgrade?
Last week I took the ARSystemInstalledConfiguration.xml that was created by the
failed installation and started a new try using it. Same errors.
In the meantime the BMC Support keeps silent…
Thomas
> On 10. Oct 2017, at 19:03, Gadgil, Abhijeet <abhijeet_gad...@bmc.com> wrote:
>
> **
> Looked the logs that were mentioned at the start of this email thread.
> We see a couple of issues within the logs and calling these out in reverse
> chronological order:
> 1. SEVERE error regarding server not starting up – this is a result of
> database changes themselves not going thru correctly in the first place.
> 2. While making database changes, there is a ORA-02437 error
> a. (Oct 06 2017 02:52:22.617 PM
> +0200),WARNING,com.bmc.install.product.arsuitekit.platforms.arsystemservers.arserver.ARServerManageUpgradeJDBCWriterErrorHandler,
> LOG EVENT {Description=[Failed to create primary key when running sql:
> ALTER TABLE ARADMIN.FILTER_PUSH ADD PRIMARY KEY ( filterId, actionIndex,
> fieldId, overlayGroup ). Upgrade will continue, but upon completion database
> should be manually corrected to ensure product performs as
> designed.],Detail=[ORA-02437: cannot validate (ARADMIN.SYS_C0062062) -
> primary key violated
> 3. Even with just this error, we would expect only some of the imports
> for FILTERs to fail. But, there are a lot many failures related to duplicate
> inserts.
>
> Looking at the log in detail, we see that database changes didn’t go through
> correctly. This is evident from the log of
> ARServerPriorToServerJDatabaseUpgradeTask class, really missing logs from
> this class tell us this class didn’t go through and inserts are failing with
> index violation error.
>
> Why did this class not execute – really either of the 2 conditions:
> a. Either the server being upgraded is secondary server – this is not
> the case and it’s confirmed in the logs.
> b. Or it did not find “featureARServer” product ID and version in
> ARSystemInstalledConfiguration.xml that existed in the install directory
> before starting the upgrade – this seems to be true in this case as Thomas
> mentioned in the mail below “ARSystemInstalledConfiguration.xml didn’t exist
> but it was created after the first failed installation. In there it says,
> server is NOT server group member. So does ar.conf”.
>
> We have seen similar issue in the past due to corrupt/incomplete or missing
> ARSystemInstalledConfiguration.xml file.
> Now why this file does not exist before upgrade is the real root cause, which
> you can check based on audit enabled on OS.
>
>
> Fix for this issue:
> · Revert the system completely back to 8.1 clean state including
> database and file system.
> · Ensure file system has correct ARSystemInstalledConfiguration.xml
> file with lines such as this:
> o <productFeature backupOnUpgrade="false" id="featureARServer"
> independentOfChildren="true" parent="featureARSystemServers"
> rebootRequiredOnInstall="false" rebootRequiredOnUninstall="false"
> rebootRequiredOnUpgrade="false" requiredDiskSpaceMode="default.windows"
> state="INSTALLED" visible="true">
> <version majorVersion="1" minorVersion="00" releaseVersion="8"/>
> …
> · Ensure there are no duplicates in FILTER_PUSH table. Use this query
> to find if there are duplicates before upgrade:
> o select filterId, actionIndex, fieldId, count(*) from filter_push group by
> filterId, actionIndex, fieldId having count(*) > 1
> o If above query finds duplicates, that needs to be fixed before starting
> the upgrade. Clearly there seem to be corrupt entries related to overlays of
> those filters. Don’t see how it will happen via the product.
>
> Then start upgrade with 9.1.03 and this time it should succeed once database
> is upgraded successfully.
>
> Regards,
> Abhijeet
>
> From: Action Request System discussion list(ARSList)
> [mailto:arslist@ARSLIST.ORG] On Behalf Of Carl Wilson
> Sent: 10 October 2017 22:19
> To: arslist@ARSLIST.ORG
> Subject: Re: Second try --> Upgrade 8.1.00 to 9.1.03 Failed
>
> **
> Hi,
> As BMC have had many issues with the 9.x installers (e.g. patches for patch
> installers), have you tried a based upgrade to 9.0/9.1 then done the 9.1 SP?
> Might be worth a try.
>
> ----------------------------------------------
>
> Kind Regards,
>
> Carl Wilson
>
>
> From: Action Request System discussion list(ARSList)
> [mailto:arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>] On Behalf Of Thomas
> Miskiewicz
> Sent: 10 October 2017 16:46
> To: arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>
> Subject: Re: Second try --> Upgrade 8.1.00 to 9.1.03 Failed
>
> **
> Carl,
>
> the pre-check tool is totally bonkers. It’s reposting misleading and false
> errors and does randomly so. From my experience it cannot be taken seriously
> in 95% cases.
>
> This time it reported this:
>
> Filter Correction Check
>
>
> Description
> In BMC Remedy AR System 8.1.02, the Ap:Alt-SetPermissions filter was
> introduced in one of the hotfixes
> (HF\pinatubo\CertifiedHotFixes\8.1.02\Multitenancy\SW00488367-ApprovalServer\MultiTenancy_Approval_Hotfix_81SP2).
> The same filter in 9.1.02 is included with an upper case 'P' in the form
> name: AP:Alt-SetPermissions. When upgrading from 8.1.02 with that hotfix to
> 9.1.02, you encounter the following error:
> "ERROR RIKMain - 382 The value(s) for this entry violate a unique index that
> has been defined for this form"
> The check validates the casing in the AP:Alt-SetPermissions filter name.
> Type
> Pre-upgrade check
> Check performed and expected result
> When upgrading from 8.1.02 with that hotfix to 9.1.02, the check detects a
> lower case 'p' in the AP:Alt-SetPermissions filter name and displays the
> following error message:
> Please correct the filter name from Ap:Alt-SetPermissions to
> AP:Alt-SetPermissions. Execute the following SQL statement to fix it:
> UPDATE filter SET name = 'AP:Alt-SetPermissions', resolvedName =
> 'AP:Alt-SetPermissions' WHERE name = 'Ap:Alt-SetPermissions';
> Corrective action
> Execute the SQL statement provided in the preceding row to update the filter
> name.
> I ran this statement and of course none records were modified. So why bother?
>
>
> Thomas
>
>
>
>
> On 10. Oct 2017, at 17:36, Carl Wilson <carlbwil...@gmail.com
> <mailto:carlbwil...@gmail.com>> wrote:
>
> **
> Hi,
> As Brian has mentioned, you could do the first server as a staged upgrade
> then for the other servers do a binary update on the remaining servers (as
> the DB will be already updated) …. It is really getting the DB information
> updated with the first run, then the binaries on the other 23 servers.
>
> Not surprising you are having issues, BMC seem to forget about people in the
> rest of the world when they create their installers and think that everyone
> runs "US English" as a default.
>
> BTW: Your install log mentions that the pre-checker failed, but there is no
> reference to what failed in the pre-check. What is in the report for this
> ('file:///ESP/usr/ar/ARSE1/Logs/result/configchecker_report_1507293439188.html
>
> <file:///ESP/usr/ar/ARSE1/Logs/result/configchecker_report_1507293439188.html>')?
>
> ----------------------------------------------
>
> Kind Regards,
>
> Carl Wilson
>
>
> From: Action Request System discussion list(ARSList)
> [mailto:arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>] On Behalf Of Thomas
> Miskiewicz
> Sent: 10 October 2017 15:55
> To: arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>
> Subject: Re: Second try --> Upgrade 8.1.00 to 9.1.03 Failed
>
> **
> I didn’t try and I won’t. I have 24 servers to upgrade!!!
>
>
>
> On 10. Oct 2017, at 16:51, Brian Pancia <panc...@finityit.com
> <mailto:panc...@finityit.com>> wrote:
>
> **
> Have you tried to upgrade one sp/version at a time instead of going directly
> to 9.1.03. I've done countless upgrades and never had a lot of luck with
> going from one version (ie 8.x to 9.x). In that case I've always did a clean
> install and migrate data. RRRChive does an excellent job at moving data.
> I've seen installers go through stating successful only to find out things
> were missing on the back end. I would recommend trying an incremental
> upgrade first and if you're spinning your tires do a clean install and
> migrate data. It might be quicker then troubleshooting the install.
>
> Brian
>
> From: Action Request System discussion list(ARSList) <arslist@ARSLIST.ORG
> <mailto:arslist@ARSLIST.ORG>> on behalf of Thomas Miskiewicz
> <tmisk...@gmail.com <mailto:tmisk...@gmail.com>>
> Sent: Tuesday, October 10, 2017 8:14:40 AM
> To: arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>
> Subject: Re: Second try --> Upgrade 8.1.00 to 9.1.03 Failed
>
> **
> No, the issue is that the installer updates the binaries and then cannot
> start the server and gives up. I also feel like giving up after the BMC
> support cannot tell us what their product does.
>
> Thomas
>
> On 10. Oct 2017, at 14:08, Saji Philip <sphili...@gmail.com
> <mailto:sphili...@gmail.com>> wrote:
>
> **
> Is this an upgrade in place? Does the installer hang (never finishes and no
> errors). We had a similar issue and the 9..x would fail ARS because we had a
> custom AL guide and it would error out on tha arcontainer table. Taking a
> backup and deleting this AL guide allowed ARS to install. Then we had
> problems with Atrium in that it didn't like our overlay on the BMC.Elements
> form. But that would error out. We just deleted the overlay. But the
> weirdest part is that ITSM would hang in installation. Putting the server to
> Allow queries to be true, even though the prechecker said otherwise, allowed
> the ITSM to go through...
> 8.1.02 going to 9.1.03
>
> On Tue, Oct 10, 2017, 6:27 AM Thomas Miskiewicz <tmisk...@gmail.com
> <mailto:tmisk...@gmail.com>> wrote:
> **
> Satya,
>
> the environment is NOT a copy or clone of another environment. The servgrp...
> Tables don’t exist.
>
> I did a clean start with deleting logs etc few times already. Nothing new.
> Same stuff in the logs.
>
> I would appreciate if you could review my armonitor.conf and
> ARSystemInstalledConfiguration.xml Maybe you see something’s I don’t.
>
> Apart from that I have no idea what else I can do. Even worse neither does
> BMC!
>
> They‘ve been discussing the matter internally since a couple of days but
> didn’t approach us with a single suggestion.
>
> When this happens (and this is how BMC support in India works 99.9% of time),
> they usually wait two weeks and send me a question whether the problem still
> persists. How ridiculous is that!
>
>
> Thomas
>
> On 6. Oct 2017, at 20:05, Satya Gandhi <satya.gan...@gmail.com
> <mailto:satya.gan...@gmail.com>> wrote:
>
> **
> Hi Thomas,
>
> A dbVersion value of 54 indicates that the database has been upgraded
>
> I also see that there is alternating information about the ARServer being
> secondary serher AT least when it is validating FTS ports
>
> If the database was a copy from another environment which was part of a
> server group, then perform the flowing steps
>
>
> Remove the old arerror log and armonitor log files and restart the arsystem
> services. If the reserves starts successfully the perform the following steps
>
> Delete records from table servgrp_board and servgrp_resources
>
> Change db version value to what is currently on prevdbversion column and
> remove the value from prevdbversion
>
> On the AR config file, set multiple ar servers to false and server group
> member to false. Also ensure that there are no references for secondary
> server for any plugins or any features. Comment those lines out, if you find
> any
>
> Remove any files from the temp directory to have clean set of install log
> files
>
> Ensure that the arsystem server licenses are valid and you are able to create
> new records in any for that already have more than 2000 entries
>
> Also set the min and max values for private queues, fast and list threads
> to the current and recommended maximum value for your infrastructure
>
> Now run the installer and let us know how it goes
>
> Thanks
> Satya Gandhi
>
> On 6 Oct 2017 6:29 pm, "Thomas Miskiewicz" <tmisk...@gmail.com
> <mailto:tmisk...@gmail.com>> wrote:
> **
> Hi,
>
> the dbVersion is 54.
>
> Server is not part of the server group, still the stupid installer says:
> server is NOT server group member, server IS server group member, server is
> NOT server group member … why?
>
> (Oct 06 2017 02:29:12.122 PM
> +0200),CONFIG,com.bmc.install.task.InstallationPropertiesHelper,
> LOG EVENT {Description=[SET PROPERTY BMC_AR_SERVER_GROUP_MEMBER],Detail=[F]}
>
> [className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Num-Preload-Schema-Segments, , 25, searchKEYandREPLACEvalue]],
> [className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Num-Preload-Threads, , 10, searchKEYandREPLACEvalue]],
> [className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Application-Enable, F, T, setCFGentry]],
> [className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Large-Result-Logging-Threshold, , 1000000, setCFGentry]],
> [className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Max-Log-History, , 8, setCFGentry]],
> [className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Max-Log-File-Size, , 134217728, setCFGentry]]]], [level=Database Only Server
> Group
> Configuration,commands=[[className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Server-Group-Member, , F, searchKEYandREPLACEvalue]]]], [level=AR Server
> Group
> Configuration,commands=[[className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Server-Group-Member, , T, searchKEYandREPLACEvalue]]]], [level=Configure
> JMX,commands=[[className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Jmx-port, , 61500, searchKEYandREPLACEvalue]]]], [level=Configure
> JMS,commands=[[className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Default-messaging-port, , 61617, searchKEYandREPLACEvalue]]]], [level=AR
> Server Atrium SSO Configurati
>
>
>
> (Oct 06 2017 02:47:07.676 PM
> +0200),INFO,com.bmc.smbu.install.common.rule.engine.StageGroup,
> LOG EVENT {Description=[Skipping execution of
> stage],Detail=[[level=Database Only Server Group
> Configuration,commands=[[className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Server-Group-Member, , F, searchKEYandREPLACEvalue]]]]]}
> (Oct 06 2017 02:47:07.676 PM
> +0200),INFO,com.bmc.smbu.install.common.rule.engine.Stage,
> LOG EVENT {Description=[Executing stage [level=AR Server Group
> Configuration,commands=[[className=com.bmc.install.product.arsuitekit.platforms.arsystemservers.task.ARServerUpdateConfigurationEntry,arguments=[/ESP/usr/ar/ARSE1/conf/ar.conf,
> Server-Group-Member, , T, searchKEYandREPLACEvalue]]]]]}
> (Oct 06 2017 02:47:07.676 PM
> +0200),INFO,com.bmc.smbu.install.common.rule.engine.extension.ExtensionCommand,
> LOG EVENT {Description=[Executing extension Command]}
>
> etc.
>
> arsystem_install_log.txt attached
>
> ARSystemInstalledConfiguration.xml didn’t exist but it was created after the
> first failed installation. In there is say, server is NOT server group
> member. So does ar.conf
>
> I’m on Solaris in this case.
>
> arerror.log has a few hundreds entries complaining that he couldn’t import
> forms:
>
> Fr Okt 06 15:26:28.836 2017 Import failed for AR System Administration: Add
> Or Remove Licenses:
> Fr Okt 06 15:26:28.953 2017 Import failed for AR System Administration:
> Display Form To Collect User Decisions:
> Fr Okt 06 15:26:29.225 2017 Import failed for AR System License: Save Produse
> Attachment:
> Fr Okt 06 15:26:29.530 2017 Import failed for AR System Licenses Console:
> Fr Okt 06 15:26:29.688 2017 Import failed for AR System: Generate License
> Usage Report:
> Fr Okt 06 15:26:29.986 2017 Import failed for AR System Licenses Audit:
> Fr Okt 06 15:26:30.229 2017 Error updating audit shadow form AR System
> Licenses Audit
>
>
>
> Thomas
>
>
>
> _ARSlist: "Where the Answers Are" and have been for 20 years_
> **
>
>
>
>
> On 6. Oct 2017, at 19:13, Satya Gandhi <satya.gan...@gmail.com
> <mailto:satya.gan...@gmail.com>> wrote:
>
> **
> Hi,
>
> Check the record in control table from a db utility
>
> Select * from control
>
> The value in dbVersion column should be 54 and if that's not the case, the
> database has not been upgraded to 9.1.03.
>
> If the AR Server binaries has been upgraded but not the database, t
> It should given an error Bout column useShaH (or something similar). This
> column is created only when the database has also been upgraded
>
> Was the server part of server group? If yes, can you please check if the
> server was considered a secondary server and hence updates only the binaries
> and not the database. This can be found on the
> systeminstalledconfiguration.xml file
>
> If you are running the server on Windows, check the productregistry.xml to
> find the install and upgrade entries for product BMC Remedy action request
> system server
>
> I cannot open the log files as I am on my mobile right now. Are there any
> glaring errors on arsystem install logs?
>
> Regards
> Satya Gandhi
>
> On 6 Oct 2017 18:06, "Grooms, Frederick W" <frederick.w.gro...@xo.com
> <mailto:frederick.w.gro...@xo.com>> wrote:
> Is there anything in the arDebug.log or arError.log?
>
>
> -----Original Message-----
> From: Action Request System discussion list(ARSList)
> [mailto:arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>] On Behalf Of Thomas
> Miskiewicz
> Sent: Friday, October 06, 2017 11:24 AM
> To: arslist@ARSLIST.ORG <mailto:arslist@ARSLIST.ORG>
> Subject: Second try --> Upgrade 8.1.00 to 9.1.03 Failed
>
> Hello Listers,
>
> at the moment I do ARS upgrades from 7.6.4, 8.1.00 and higher to 9.1.03 for
> various customers. OS: Linux and Solaris.
>
> Not a single system that just worked! I'm really not sure how to move to
> production under this circumstances...
>
> What's even worse: the BMC support in India has zero clue and just keeping us
> busy with their ideas and requests. In all the month not a single hint with
> substance! BTW, talked to some partners and customers which I do not work for
> - same issues! But BMC support always pretends they never heard of any issues.
>
> The logs of my recent
> installation:https://s3.eu-central-1.amazonaws.com/miskiewicz/arslist/INC112333.tar.gz
> <https://s3.eu-central-1.amazonaws.com/miskiewicz/arslist/INC112333.tar.gz>
> Maybe one of you has an idea what is going on.
>
> Basically the job went like this:
>
> 1. Clean and working 8.1.00 installation.
> 2. Installer 9.1.03 runs.
> 3. Installer 9.1.03 cannot start the server it installed and gives up.
> 4. Result: A messed up ARS 9.1.03 installation that doesn't start, but is
> trying to do a gazzilion of inserts into various tables and is violating the
> unique index at the same time.
>
> I restored and started from scratch. Same result. Not sure what to do. The
> only comfort is BMC doesn't know either. I feel less stupid.
>
>
> Thomas
>
> _______________________________________________________________________________
> UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
> <http://www.arslist.org/>
> "Where the Answers Are, and have been for 20 years"
> _ARSlist: "Where the Answers Are" and have been for 20 years_
>
> _ARSlist: "Where the Answers Are" and have been for 20 years_
> _ARSlist: "Where the Answers Are" and have been for 20 years_
> _ARSlist: "Where the Answers Are" and have been for 20 years_
> _ARSlist: "Where the Answers Are" and have been for 20 years_
> _ARSlist: "Where the Answers Are" and have been for 20 years_
> DISCLAIMER: The information contained in this e-mail and its attachments
> contain confidential information belonging to the sender, which is legally
> privileged. The information is intended only for the use of the recipient(s)
> named above. If you are not the intended recipient, you are notified that any
> disclosure, copying, distribution or action in reliance upon the contents of
> the information transmitted is strictly prohibited. If you have received this
> information in error, please delete it immediately. _ARSlist: "Where the
> Answers Are" and have been for 20 years_
>
> _ARSlist: "Where the Answers Are" and have been for 20 years_
>
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
>
> Virus-free. www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
> _ARSlist: "Where the Answers Are" and have been for 20 years_
>
> _ARSlist: "Where the Answers Are" and have been for 20 years_
> _ARSlist: "Where the Answers Are" and have been for 20 years_
> _ARSlist: "Where the Answers Are" and have been for 20 years_
_______________________________________________________________________________
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
"Where the Answers Are, and have been for 20 years"
