[Veritas-bu] Strange status 13 during bpstart script
Guys i've been reading on Status 13 quite a bit but am still flummoxed as to why we are getting this on an environment here. Ok Netbackup 6.0MP4 Windows 2003 Master server Netbackup 6.0MP4 Windows 2003 Media server attached to EMC disk virtual storage. Netbackup 6.0MP4 Linux Client running Suse linux Basically we run a bpstart script which waits for a flag to appear from another script ran from cron. Once the flag is found we run a 3rd party application script to shut down some applications. Occasionally these 3rd party app shutdown scripts dont finish properly and dont return to our bpstart script so it just sits there. After the bpstart time out is reached the bpbkar on the client shows a status 74 and exits. Now the weird bit. On the master server after 2 hours of the job starting a Status13 is reported that the bpbrm process was forcibly terminated on the client. The job then sits as active in the activity monitor sometimes, and other times finished with status 13. As this isn't a windows client vsp and vss are not to blame. There are firewalls between media servers and clients though, so i was wondering if some network issue was to blame??? Any pointers welcomed Cheers ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Issues with Oracle backup Netbackup 5.1
Are the oracle backups being issue from an agent such as RMAN? I know in the past the number of simultaneous backup streams are actually pushed from oracle and not netbackup. This was all set up in the RMAN scripts to stream 4 sessions at a time. This has no bearing on me having 8 drives available. Maybe chat to the DBA's Cheers judy_hinchcli...@administaff.com wrote: Nope, it was not there in 5.1 So you have 8 drives - and your storage unit is set to use 8 drives... correct? And multiplexing for the storage unit is set to 4 - so you should be able to have 32 jobs running at once - 4 to each of the 8 drives. 4 x 8 = 32 So let's look at something else. Are you running more than one job per client? (multistreamed backups) If so lets look at how many jobs per client are allowed. Master server properties, Global Attributes Maximum jobs per client -if this is one or 2 look at what jobs are running and which ones are queue. Has that Client reached it's max number of concurrently running jobs? The Maximum jobs per client property specifies the maximum number of backup and archive jobs that NetBackup clients can perform concurrently. Default: 1 job. This can also be set per client - so if the Global is like 8 but you still have queue, again in the master properties look at Client Attributes See if a client is listed here, if yes then look at the General tab and see if the client has be set to limit the number of data streams The Maximum data streams property specifies the maximum number of jobs that are allowed at one time for each selected client. (This value applies to the number of jobs on the client, even if multistreaming is not used.) -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Sushil Sent: Thursday, May 13, 2010 3:30 AM To: VERITAS-BU@MAILMAN.ENG.AUBURN.EDU Subject: [Veritas-bu] Issues with Oracle backup Netbackup 5.1 First in the gui add the column to your view state details That will tell you why a job is not running. Could be you have on your storage unit, max drives, or max jobs and if those have been met then the job will queue. sorry but i could not find state details as column in activity monitor , maybe its not there in 5.1. Also there is no value set on Max. Conc. Jobs in storage unit. Is your storage unit definitely configured to use all the drives? And this may seem like a strange thing to ask but what is your client_connect_timeout set to on your master server? Yes it is configured to use all the drives. The client_connect_timeout value is 4000 seconds. Another thing which i noticed is that Maximum multiplexing per drive is set as 4 on storage unit. But multiplexing is set as 16 on policies. Could this be the reason for queued jobs? But at times i have noticed that 1 out of 8 drives is utilised and still there are approx 5-6 jobs in queued state and when i see job details there is nothing. +-- |This was sent by sushil.gamb...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] Issues with Oracle backup Netbackup 5.1
So let's look at something else. Are you running more than one job per client? (multistreamed backups) If so lets look at how many jobs per client are allowed. Master server properties, Global Attributes Maximum jobs per client -if this is one or 2 look at what jobs are running and which ones are queue. Has that Client reached it's max number of concurrently running jobs? The Maximum jobs per client property specifies the maximum number of backup and archive jobs that NetBackup clients can perform concurrently. Default: 1 job. This can also be set per client - so if the Global is like 8 but you still have queue, again in the master properties look at Client Attributes See if a client is listed here, if yes then look at the General tab and see if the client has be set to limit the number of data streams The Maximum data streams property specifies the maximum number of jobs that are allowed at one time for each selected client. (This value applies to the number of jobs on the client, even if multistreaming is not used.) This is oracle database backup and multistream at client side is set to 4. The Maximum jobs per client value is set to 10. Strange thing is not even the single stream/job of this client is active when the backup is started all of them are queued for 1 or 2 hours before they become active. This client is hosted on solaris and there are two database running on single physical server and both of the database hava seperate logical hostname and are registered seperately on Netbackup. Logically they should be treated as different clients(?) Is there any kind of tracing we can enable on client to have some extra details which may help? Thanks, Sushil +-- |This was sent by sushil.gamb...@gmail.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] RMAN crosscheck
My DBAs are starting to question me about an RMAN crosscheck they are running. Essentially I gather that when they ran it the crosscheck seemed to report even backups run in the last 2 days as expired. On checking the Data Domain unit I can see the images are still there and on running NetBackup commands I see these are NOT expired from NBU's perspective. On doing a search I did find a document at Symantec that talked about RMAN expirations but it only went up through 6.0 so I'm not sure if it is still valid for 6.5. It says essentially that on the NBU side we should set very long retentions (e.g. INFINITY) for all RMAN backup policies then let RMAN keep track of retentions itself. The downside I see to this is we do vaulting of the images on data domain to tape - we set retention on data domain to 1 month then the vault copies get longer retentions (e.g. 3 months for a daily backup). How would we get RMAN to set and keep track of such retention differences? The DBAs have opened a TAR with Oracle to see why the crosscheck is reporting the images as expired but I suspect from the NBU document that the answer will be something like it is expired so far as RMAN is concerned. This also begs the question as to whether RMAN could be used to restore the backups even if they aren't expired so far as NBU is concerned. Does anyone know the answer to that? Proud partner. Susan G. Komen for the Cure. Please consider our environment before printing this e-mail or attachments. -- CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you. -- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] VMware clients
On 5/13/2010 4:05 PM, Victor Engle wrote: Just wondered if anyone could discuss the best options for backing up VMs running on ESX 3.5 and 4.0 systems? What are the pros and cons and which methods allow for file level restores for the VM. My backup server is solaris. Thanks, Vic ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu I am waiting until we get our infrastructure to vSphere 4.0 and NBU 7 to do vStorage backups which seem to be da bomb digity. All the ducks just got lined up in a row for this just recently, but I'll still wait until June to make sure nothing comes out in the field from other people. DataDomain's DDBoost just came out which is their new OST plugin which supports NBU 7. It also does some media server side de-duplication processing which I am hoping leads the way to global de-dupe and client side de-dupe. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Strange status 13 during bpstart script
Yeah sorry perhaps not explained correctly. The bpstart time out is 8 hours (28800) The status 74 is what we would expect to see on the Master Server job view, and is actually what the CLIENT bpbkar log shows after 8 hours of the job hanging. However, the issue is the Master server is reporting Status 13 after 2 hours of inactivity, i.e whilst the bpstart job is running on the client and BEFORE the timeout is reached. Cheers Whelan, Patrick wrote: I'm not quite sure what you are asking. Obviously the BPSTART_TIMEOUT needs to be increased. If the error 74s stop then the error 13s should go away. Or am I missing the point altogether? Regards, Patrick Whelan NetBackup Specialist Wholesale Markets and Treasury Trading Lloyds Banking Group Desk: +44 (0) 207 158 6123 Loc: OBS 2C-132 P please don't print this e-mail unless you really need to. -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Dave Markham Sent: 14 May 2010 12:37 To: veritas-bu@mailman.eng.auburn.edu Subject: [Veritas-bu] Strange status 13 during bpstart script Guys i've been reading on Status 13 quite a bit but am still flummoxed as to why we are getting this on an environment here. Ok Netbackup 6.0MP4 Windows 2003 Master server Netbackup 6.0MP4 Windows 2003 Media server attached to EMC disk virtual storage. Netbackup 6.0MP4 Linux Client running Suse linux Basically we run a bpstart script which waits for a flag to appear from another script ran from cron. Once the flag is found we run a 3rd party application script to shut down some applications. Occasionally these 3rd party app shutdown scripts dont finish properly and dont return to our bpstart script so it just sits there. After the bpstart time out is reached the bpbkar on the client shows a status 74 and exits. Now the weird bit. On the master server after 2 hours of the job starting a Status13 is reported that the bpbrm process was forcibly terminated on the client. The job then sits as active in the activity monitor sometimes, and other times finished with status 13. As this isn't a windows client vsp and vss are not to blame. There are firewalls between media servers and clients though, so i was wondering if some network issue was to blame??? Any pointers welcomed Cheers ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu This e-mail is private and confidential and may contain privileged material. If you have received this e-mail in error, please notify the sender and delete it immediately. You must not copy, distribute, disclose or use any of the information in it or any attachments. Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland, number 95000. Telephone: 0131 225 4555. Lloyds TSB Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales, number 2065. Telephone: 020 7626 1500. Lloyds TSB Scotland plc. Registered Office: Henry Duncan House, 120 George Street, Edinburgh EH2 4LH. Registered in Scotland, number 95237. Telephone: 0131 225 4555. Cheltenham Gloucester plc. Registered Office: Barnett Way, Gloucester GL4 3RL. Registered in England and Wales, number 2299428. Telephone: 01452 372372. Cheltenham Gloucester Savings is a division of Lloyds TSB Bank plc. Lloyds TSB Bank plc, Lloyds TSB Scotland plc and Cheltenham Gloucester plc are authorised and regulated by the Financial Services Authority. Telephone calls may be monitored or recorded. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __ ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Strange status 13 during bpstart script
Well on the client they are empty and on the master server windows its using the vxlog thingy which i've not looked at before. I.e there's nothing in ~netbackup\logs\bpbrm. :( I'm a unix engineer so find windows very frustrating as regards looking and searching through logs. cheers Whelan, Patrick wrote: What do the brbrm logs say? Regards, Patrick Whelan NetBackup Specialist Wholesale Markets and Treasury Trading Lloyds Banking Group Desk: +44 (0) 207 158 6123 Loc: OBS 2C-132 P please don't print this e-mail unless you really need to. -Original Message- From: Dave Markham [mailto:dave.mark...@fjserv.net] Sent: 14 May 2010 14:30 To: Whelan, Patrick Subject: Re: [Veritas-bu] Strange status 13 during bpstart script No it's just set to 900 still. We could increase that, but if it's the CLIENT_READ_TIMEOUT why is that not being hit and exiting sooner. Looking at the history when there is a problem, its reporting out status 13 and this brbrm forcible shutdown after 2hours each time Cheers Whelan, Patrick wrote: Have you increased the CLIENT_READ_TIMEOUT parameter? Regards, Patrick Whelan NetBackup Specialist Wholesale Markets and Treasury Trading Lloyds Banking Group Desk: +44 (0) 207 158 6123 Loc: OBS 2C-132 P please don't print this e-mail unless you really need to. -Original Message- From: Dave Markham [mailto:dave.mark...@fjserv.net] Sent: 14 May 2010 13:40 To: Whelan, Patrick; veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Strange status 13 during bpstart script Yeah sorry perhaps not explained correctly. The bpstart time out is 8 hours (28800) The status 74 is what we would expect to see on the Master Server job view, and is actually what the CLIENT bpbkar log shows after 8 hours of the job hanging. However, the issue is the Master server is reporting Status 13 after 2 hours of inactivity, i.e whilst the bpstart job is running on the client and BEFORE the timeout is reached. Cheers Whelan, Patrick wrote: I'm not quite sure what you are asking. Obviously the BPSTART_TIMEOUT needs to be increased. If the error 74s stop then the error 13s should go away. Or am I missing the point altogether? Regards, Patrick Whelan NetBackup Specialist Wholesale Markets and Treasury Trading Lloyds Banking Group Desk: +44 (0) 207 158 6123 Loc: OBS 2C-132 P please don't print this e-mail unless you really need to. -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of Dave Markham Sent: 14 May 2010 12:37 To: veritas-bu@mailman.eng.auburn.edu Subject: [Veritas-bu] Strange status 13 during bpstart script Guys i've been reading on Status 13 quite a bit but am still flummoxed as to why we are getting this on an environment here. Ok Netbackup 6.0MP4 Windows 2003 Master server Netbackup 6.0MP4 Windows 2003 Media server attached to EMC disk virtual storage. Netbackup 6.0MP4 Linux Client running Suse linux Basically we run a bpstart script which waits for a flag to appear from another script ran from cron. Once the flag is found we run a 3rd party application script to shut down some applications. Occasionally these 3rd party app shutdown scripts dont finish properly and dont return to our bpstart script so it just sits there. After the bpstart time out is reached the bpbkar on the client shows a status 74 and exits. Now the weird bit. On the master server after 2 hours of the job starting a Status13 is reported that the bpbrm process was forcibly terminated on the client. The job then sits as active in the activity monitor sometimes, and other times finished with status 13. As this isn't a windows client vsp and vss are not to blame. There are firewalls between media servers and clients though, so i was wondering if some network issue was to blame??? Any pointers welcomed Cheers ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu This e-mail is private and confidential and may contain privileged material. If you have received this e-mail in error, please notify the sender and delete it immediately. You must not copy, distribute, disclose or use any of the information in it or any attachments. Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland, number 95000. Telephone: 0131 225 4555. Lloyds TSB Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales, number 2065. Telephone: 020 7626 1500. Lloyds
[Veritas-bu] VMware clients
Vstorage does not scale well at all. We had it backing up around 1000 vm clients and crashed Vcenter everytime. David Stanaway wrote: On 5/13/2010 4:05 PM, Victor Engle wrote: Just wondered if anyone could discuss the best options for backing up VMs running on ESX 3.5 and 4.0 systems? What are the pros and cons and which methods allow for file level restores for the VM. My backup server is solaris. Thanks, Vic ___ Veritas-bu maillist - Veritas-bu at mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu I am waiting until we get our infrastructure to vSphere 4.0 and NBU 7 to do vStorage backups which seem to be da bomb digity. All the ducks just got lined up in a row for this just recently, but I'll still wait until June to make sure nothing comes out in the field from other people. DataDomain's DDBoost just came out which is their new OST plugin which supports NBU 7. It also does some media server side de-duplication processing which I am hoping leads the way to global de-dupe and client side de-dupe. ___ Veritas-bu maillist - Veritas-bu at mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu +-- |This was sent by josh.op...@salliemae.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] VMware clients
1000 clients simultaneously? Across how many hosts? Did you troubleshoot at all with VMware support? -Jonathan -Original Message- From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of jopper Sent: Friday, May 14, 2010 11:12 AM To: VERITAS-BU@MAILMAN.ENG.AUBURN.EDU Subject: [Veritas-bu] VMware clients Vstorage does not scale well at all. We had it backing up around 1000 vm clients and crashed Vcenter everytime. David Stanaway wrote: On 5/13/2010 4:05 PM, Victor Engle wrote: Just wondered if anyone could discuss the best options for backing up VMs running on ESX 3.5 and 4.0 systems? What are the pros and cons and which methods allow for file level restores for the VM. My backup server is solaris. Thanks, Vic ___ Veritas-bu maillist - Veritas-bu at mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu I am waiting until we get our infrastructure to vSphere 4.0 and NBU 7 to do vStorage backups which seem to be da bomb digity. All the ducks just got lined up in a row for this just recently, but I'll still wait until June to make sure nothing comes out in the field from other people. DataDomain's DDBoost just came out which is their new OST plugin which supports NBU 7. It also does some media server side de-duplication processing which I am hoping leads the way to global de-dupe and client side de-dupe. ___ Veritas-bu maillist - Veritas-bu at mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu +-- |This was sent by josh.op...@salliemae.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] VMware clients
[quote=Martin, Jonathan]1000 clients simultaneously? Across how many hosts? Did you troubleshoot at all with VMware support? -Jonathan Not all at once. We were able to crash Vcenter with only 21 clients running at once. We were using 1 master and 2 media servers. We have had a case open with VMware and Symantec. The issue is with the number of open cifs connections. Vcenter does not always close these sessions. VMware says this is a known issue and they are working on it. Symantec has also been able to recreate the issue. They are working on a solution. +-- |This was sent by josh.op...@salliemae.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] last update ARB - Automated Runbook
Mr. Rockey, Who is the Automated Runbook´s author? Thanks, Márcia Garcia Symantec Certified Professional ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
