Good to hear. I opened a ticket as Andy recommended but have not heard anything about an available eFix.
FWIW, I seemed to have caused these issues when I re-activated replication on one of my servers. This server was originally using virtual volumes to create offsite backups (trying to move away from tapes since we will be getting rid of magnetic tape/libraries by the time we move to a new datacenter by the end of 2023). However, we started having weekly cores/crashes on this server. At first IBM gave us an eFix (8.1.14.102) but we continued to have weekly crashes and when IBM analyzed our latest crash, they said they discovered a new problem caused by RECONCILE VOLUMES which we use daily on these virtual volumes/server. On Wed, Jul 20, 2022 at 4:58 AM Loon, Eric van (ITOP DI) - KLM < eric-van.l...@klm.com> wrote: > Hi everybody, > > I tested a fix I received from IBM this morning, which seems to be working > OK. So let's hope IBM will release a 8.1.15.100 soon. > > Kind regards, > Eric van Loon > Core Infra > > -----Original Message----- > From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> On Behalf Of Andrew > Raibeck > Sent: dinsdag 19 juli 2022 22:46 > To: ADSM-L@VM.MARIST.EDU > Subject: Re: 8.1.15 Linux client update causing failures > > Hi, the following flash is published for this issue. > > https://www.ibm.com/support/pages/node/6605073 > > Best regards, > > Andy > > Andrew Raibeck > IBM Spectrum Protect Level 3 > IBM Systems, Storage > stor...@us.ibm.com > > IBM > > -----Original Message----- > From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> On Behalf Of Andrew > Raibeck > Sent: Tuesday, 19 July, 2022 07:05 > To: ADSM-L@VM.MARIST.EDU > Subject: [EXTERNAL] Re: 8.1.15 Linux client update causing failures > > Hi Michael, > > The APAR is IT41452. It is not yet visible on the web, but it should be in > the next day or two. Then, until the APAR is closed, you need to log in > with your IBM ID to view it. > > Best regards, > > Andy > > Andrew Raibeck > IBM Spectrum Protect Level 3 > IBM Systems, Storage > stor...@us.ibm.com > > IBM > > -----Original Message----- > From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> On Behalf Of Michael > Roesch > Sent: Tuesday, 19 July, 2022 05:30 > To: ADSM-L@VM.MARIST.EDU > Subject: [EXTERNAL] Re: 8.1.15 Linux client update causing failures > > Hi all, > > Did IBM already create an APAR for this issue? We're currently holding off > on the 8.1.15.0 client rollout and it would be great to know if IBM's > initial assumptions have proven true, which would mean we're not affected > b/c we don't use replication. > > Thank you > > Kind regards > Michael Roesch > > On Thu, Jul 14, 2022 at 2:29 PM Andrew Raibeck <stor...@us.ibm.com> wrote: > > > Hi Zoltan, > > > > The problem I refer to occurs immediately while starting the client. > > Even by merely running "dsmc", it crashes. Is that the case for you? > > Or does the operation run for a period of time, and then crash? If the > > problem occurs on a particular client, does it always occur? Or is it > intermittent? > > > > If you have gdb installed on a representative system, then try getting > > a stack trace. > > > > 1. Start gdb: > > > > gdb /opt/tivoli/tsm/client/ba/bin/dsmc core_file_name > > > > where "core_file_name" is the name of the core. > > > > 2. In the "(gdb)" prompt, type these commands: > > > > bt > > q > > > > The first command shows the backtrace (which is the part of interest) > > and the second command quits the debugger. > > > > If you do not have a core file, then it is probably because the core > > file ulimit is too small or 0. Run "ulimit -c" to display the current > value. > > Then run "ulimit -c unlimited", reproduce the issue, and get the > > backtrace as I previously mentioned. After, you can set the ulimit > > back to its original value and delete the core file. > > > > The following article includes an alternative (and more comprehensive) > > doc > > collection: > > > > > > https://www.ibm.com/support/pages/collecting-data-spectrum-protect-cli > > ent-crash-linux > > > > The following is a representative backtrace of the problem I referred to. > > > > (gdb) bt > > #0 0x00007f04e2c06387 in raise () from /lib64/libc.so.6 > > #1 0x00007f04e2c07a78 in abort () from /lib64/libc.so.6 > > #2 0x0000000000712a74 in psTrapHandler(int) () > > #3 <signal handler called> > > #4 0x00007f04e2c06387 in raise () from /lib64/libc.so.6 > > #5 0x00007f04e2c07a78 in abort () from /lib64/libc.so.6 > > #6 0x00007f04e2c48ed7 in __libc_message () from /lib64/libc.so.6 > > #7 0x00007f04e2c51299 in _int_free () from /lib64/libc.so.6 > > #8 0x00007f04e2c3e1b7 in fclose@@GLIBC_2.2.5 () from /lib64/libc.so.6 > > #9 0x00000000006efe2f in clientOptions::optSaveReplConnInfo() () > > #10 0x000000000068103f in cuSignOnEResp(Sess_o*) () > > #11 0x0000000000652b75 in scSignOnTheSession(Sess_o*) () > > #12 0x0000000000658ce3 in NegotiateSession(Sess_o*) () > > #13 0x0000000000653648 in OpenSess(Sess_o*, bool) () > > #14 0x0000000000654ad0 in Logon(Sess_o*) () > > #15 0x0000000000656c36 in CheckSession(Sess_o*, sessLoadPolicy_t) () > > #16 0x000000000043e5fe in dscInit(int, char**, cliType_t) () > > #17 0x000000000043eb5b in dscmain(int, char**) () > > #18 0x000000000043b3fa in main () > > > > If your backtrace differs, or you want further assistance diagnosing > > your particular instance of the crash, then please open a support case > with IBM. > > If you want, you can privately email your case number to me, for my > > awareness. > > > > If you open a case with IBM, it will help support "hit the ground > running" > > if you can provide some information up front. > > > > 1. Temporarily enable core file creation. > > > > 2. Add these options to configure the client for SERVICE tracing: > > > > TRACEFLAGS SERVICE > > TRACEFILE /some_path/dsmc_trace.txt > > > > (I'm sure you have done this befoe.) > > > > 3. Reproduce the problem. > > > > 4. Collect the diagnostic data as described in this article: > > > > > > https://www.ibm.com/support/pages/collecting-data-spectrum-protect-cli > > ent-crash-linux > > > > 5. Put the core file ulimit setting back to its original value. > > > > 6. Open a case with IBM and provide the service trace, dsmc output (or > > dsmsched.log if you schedule the operation), dsmerror.log, and the > > files collected from the article of step 4. > > > > You can remove the core file after the data is sent to IBM. > > > > Regards, > > > > Andrew Raibeck > > IBM Spectrum Protect Level 3 > > IBM Systems, Storage > > stor...@us.ibm.com > > > > IBM > > > > -----Original Message----- > > From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> On Behalf Of > > Zoltan Forray > > Sent: Thursday, 14 July, 2022 08:04 > > To: ADSM-L@VM.MARIST.EDU > > Subject: [EXTERNAL] Re: 8.1.15 Linux client update causing failures > > > > Andy, > > > > You said *"- The client is configured for automatic failover to a > > replication server"* but that doesn't jive with our > situation/environment. > > Until recently, we have not run replication on any of our backups > > (offsite copies are to tape or virtual volumes on another ISP server) > > and disk backups are to non-container (FILE) storage. I queried our > > Linux admins and these failures are all over the place - not just for > > the few we recently started replicating. > > > > On Thu, Jul 14, 2022 at 6:39 AM Andrew Raibeck <stor...@us.ibm.com> > wrote: > > > > > Hi, > > > > > > Initial assessment is that clients that meet all of the following > > > conditions are affected: > > > > > > - The client is Linux (any architecture), including backup-archive > > > client and API applications > > > - The client is configured for automatic failover to a replication > > > server > > > > > > Note that the preceding information is subject to change as we learn > > more. > > > > > > Kindest regards, > > > > > > Andy > > > > > > Andrew Raibeck > > > IBM Spectrum Protect Level 3 > > > IBM Systems, Storage > > > stor...@us.ibm.com > > > > > > IBM > > > > > > -----Original Message----- > > > From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> On Behalf Of > > > Michael Roesch > > > Sent: Thursday, 14 July, 2022 06:23 > > > To: ADSM-L@VM.MARIST.EDU > > > Subject: [EXTERNAL] Re: 8.1.15 Linux client update causing failures > > > > > > Hi, > > > > > > Does the client for AIX show this problem as well? Did this happen > > > on > > > x64 Linux or also on PPC? > > > > > > Kind regards > > > Michael Roesch > > > > > > On Wed, Jul 13, 2022 at 4:19 AM Saravanan Palanisamy < > > > evergreen.sa...@gmail.com> wrote: > > > > > > > So far We don’t see any issues in windows. All scheduled backups > > > > are successful. > > > > > > > > Regards > > > > Sarav > > > > +65 9857 8665 > > > > > > > > > > > > > On 13 Jul 2022, at 9:00 AM, Tom Alverson > > > > > <tom.alver...@gmail.com> > > > wrote: > > > > > > > > > > Is the windows client safe for this version? > > > > > > > > > > > > > > >> On Tue, Jul 12, 2022, 7:06 PM Saravanan Palanisamy < > > > > >> evergreen.sa...@gmail.com> wrote: > > > > >> > > > > >> Hi Zoltan > > > > >> > > > > >> Yes we have also faced same issue and IBM support confirmed > > > > >> they will > > > > open > > > > >> new APAR. We have reverted all clients v8.1.14 to address the > issue. > > > > >> > > > > >> Regards > > > > >> Sarav > > > > >> +65 9857 8665 > > > > >> > > > > >> > > > > >>>> On 13 Jul 2022, at 2:31 AM, Del Hoobler <hoob...@us.ibm.com> > > wrote: > > > > >>> > > > > >>> Hi Zoltan, > > > > >>> > > > > >>> IBM knows about this ... and it's being actively worked. > > > > >>> > > > > >>> I am sure it will be mitigated soon. > > > > >>> > > > > >>> Feel free to open a fresh IBM Support case so you can be > > > > >>> quickly > > > > >> notified once it is fixed. > > > > >>> > > > > >>> > > > > >>> Del > > > > >>> > > > > >>> ________________________________ > > > > >>> From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> on behalf > > > > >>> of > > > > >> Zoltan Forray <zfor...@vcu.edu> > > > > >>> Sent: Tuesday, July 12, 2022 1:56 PM > > > > >>> To: ADSM-L@VM.MARIST.EDU <ADSM-L@VM.MARIST.EDU> > > > > >>> Subject: [EXTERNAL] 8.1.15 Linux client update causing > > > > >>> failures > > > > >>> > > > > >>> We started rolling out the 8.1.15 Linux client via apogee and > > > > >>> many > > > > >> clients > > > > >>> are failing with: > > > > >>> > > > > >>> *** Error in `dsmc': double free or corruption (!prev): > > > > >> 0x0000000002baf970 > > > > >>> *** > > > > >>> > > > > >>> The only matching Google hit goes back to a V7 client issue - > > > > >>> long > > > > fixed. > > > > >>> > > > > >>> At first I thought it might be related to a Linux flavor (we > > > > >>> run > > > > Rocky8, > > > > >>> CentOS 7.1/8, RHEL 7/8) but I don't see a pattern. All of > > > > >>> these > > > > servers > > > > >>> are upgrading from 8.1.14 and downleveling the client to > > > > >>> 8.1.14 fixes > > > > >> this. > > > > >>> > > > > >>> Anyone else seeing this? > > > > >>> > > > > >>> FWIW, I could swear that during one of my many Google searches > > > > >>> I saw a reference to 8.1.15.2 (eFix?) but now I can't find it > again. > > > > >>> -- > > > > >>> *Zoltan Forray* > > > > >>> Enterprise Backup Administrator VMware Systems Administrator > > > > >>> Enterprise Compute & Storage Platforms Team VCU Infrastructure > > > > >>> Services www.ucc.vcu.edu<INVALID URI REMOVED > > > > >>> __www.ucc.vcu.edu&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ij6DLy1l > > > > >>> 7w > > > > >>> Dp > > > > >>> CbTfcDkLC_KknvhyGdCy_RnAGnhV37I&m=lx_phvPLtz3INd_W8oJ4eGdfN5WY > > > > >>> sI > > > > >>> tt > > > > >>> NeFE_AVOLSP0Mp2Rq34RX6Nb9ogoyHYd&s=e_k1hOLaA6N1PWcNH0bBFZaIJtG > > > > >>> 1F Ab mQyN9dy5iR6c&e= > zfor...@vcu.edu - 804-828-4807 Don't > > > > >>> be a phishing victim - VCU and other reputable organizations > > > > >>> will never use email to request that you reply with your > > > > >>> password, social security number or confidential personal > > > > >>> information. For more details visit INVALID URI REMOVED > > > > >>> du_&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ij6DLy1l7wDpCbTfcDkLC_ > > > > >>> Kk > > > > >>> nv > > > > >>> hyGdCy_RnAGnhV37I&m=lx_phvPLtz3INd_W8oJ4eGdfN5WYsIttNeFE_AVOLS > > > > >>> P0 > > > > >>> Mp > > > > >>> 2Rq34RX6Nb9ogoyHYd&s=VS499q-uVPEP-6-QrjIgicZ_oLB4kvyLNBmMlJeWB > > > > >>> EI > > > > >>> &e > > > > >>> = > > > > >>> <INVALID URI REMOVED. > > > > >>> questionpro.com&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ij6DLy1l7w > > > > >>> Dp > > > > >>> Cb > > > > >>> TfcDkLC_KknvhyGdCy_RnAGnhV37I&m=lx_phvPLtz3INd_W8oJ4eGdfN5WYsI > > > > >>> tt > > > > >>> Ne > > > > >>> FE_AVOLSP0Mp2Rq34RX6Nb9ogoyHYd&s=BqXWlinKYJb1mlgHqBJJz4HitGo2C > > > > >>> CE > > > > >>> Et > > > > >>> UAK46yC9F4&e= > > > > > >> > > > > > > > > > > > > > -- > > *Zoltan Forray* > > Enterprise Backup Administrator > > VMware Systems Administrator > > Enterprise Compute & Storage Platforms Team VCU Infrastructure > > Services www.ucc.vcu.edu zfor...@vcu.edu - 804-828-4807 Don't be a > > phishing victim > > - VCU and other reputable organizations will never use email to > > request that you reply with your password, social security number or > > confidential personal information. For more details visit INVALID URI > > REMOVED > > d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ij6DLy1l7wDpCbTfcDkLC_KknvhyGdCy_R > > nAGnhV37I&m=EEuklCYkXYF-yd7jHVsAJ8zTkqxPPwPcAEUJXA6tQpb9SoI4zqv3Mpf5kP > > hzANx0&s=oNmArqVXVfujOL5fee2DdAfVkQIs3INUKnoP1zbExhQ&e= > > <INVALID URI REMOVED > > tionpro.com&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=Ij6DLy1l7wDpCbTfcDkLC_ > > KknvhyGdCy_RnAGnhV37I&m=EEuklCYkXYF-yd7jHVsAJ8zTkqxPPwPcAEUJXA6tQpb9So > > I4zqv3Mpf5kPhzANx0&s=b6vXwh15EQbCaGcNAUkGGGZ7norjBvl9Gf_fsXR4cHk&e= > > > > ******************************************************** > For information, services and offers, please visit our web site: > http://www.klm.com. This e-mail and any attachment may contain > confidential and privileged material intended for the addressee only. If > you are not the addressee, you are notified that no part of the e-mail or > any attachment may be disclosed, copied or distributed, and that any other > action related to this e-mail or attachment is strictly prohibited, and may > be unlawful. If you have received this e-mail by error, please notify the > sender immediately by return e-mail, and delete this message. > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its > employees shall not be liable for the incorrect or incomplete transmission > of this e-mail or any attachments, nor responsible for any delay in receipt. > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch > Airlines) is registered in Amstelveen, The Netherlands, with registered > number 33014286 > ******************************************************** > -- *Zoltan Forray* Enterprise Backup Administrator VMware Systems Administrator Enterprise Compute & Storage Platforms Team VCU Infrastructure Services www.ucc.vcu.edu zfor...@vcu.edu - 804-828-4807 Don't be a phishing victim - VCU and other reputable organizations will never use email to request that you reply with your password, social security number or confidential personal information. For more details visit http://phishing.vcu.edu/ <https://adminmicro2.questionpro.com>
