Re: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4?
Doug, We have also seen these errors occur in the manner you describe. We traced it to the Pempersist issue, which Veritas support confirmed WAS NOT fixed in MP3 OR MP4 as they originally posted in the MP4 notes. Each maintenance pack claimed the next was going to fix the issue. Sigh... See the following Tech Notes for more info: MP1 http://seer.entsupport.symantec.com/docs/281780.htm MP2 http://seer.entsupport.symantec.com/docs/283180.htm MP3 http://seer.entsupport.symantec.com/docs/285529.htm MP4 http://seer.entsupport.symantec.com/docs/285941.htm It is supposedly fixed in MP5, but I've yet to get confirmation on this from anyone out in the field that had experienced the issue MP5 http://seer.entsupport.symantec.com/docs/290387.htm If you are running anything below MP5: and, if you pempersist file, which is in the following directory in a windows environment, Install Path\Program Files\VERITAS\NetBackup\bin\bpsched.d\, gets above about 260K in size, you'll likely start to see odd behavior/errors from the scheduler; not all of which readily appear related to the scheduler. A prime example is the status 41 errors. Good luck. Kent Eagle MTS Infrastructure Engineer II, MCP, MCSE Tech Services / SMSS Visit our website at www.wilmingtontrust.com Investment products are not insured by the FDIC or any other governmental agency, are not deposits of or other obligations of or guaranteed by Wilmington Trust or any other bank or entity, and are subject to risks, including a possible loss of the principal amount invested. This e-mail and any files transmitted with it may contain confidential and/or proprietary information. It is intended solely for the use of the individual or entity who is the intended recipient. Unauthorized use of this information is prohibited. If you have received this in error, please contact the sender by replying to this message and delete this material from any system it may be on. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4?
On Wed, 29 Aug 2007, Eagle, Kent wrote: Doug, We have also seen these errors occur in the manner you describe. We traced it to the Pempersist issue, which Veritas support confirmed WAS NOT fixed in MP3 OR MP4 as they originally posted in the MP4 notes. Each maintenance pack claimed the next was going to fix the issue. Sigh... See the following Tech Notes for more info: MP1 http://seer.entsupport.symantec.com/docs/281780.htm MP2 http://seer.entsupport.symantec.com/docs/283180.htm MP3 http://seer.entsupport.symantec.com/docs/285529.htm MP4 http://seer.entsupport.symantec.com/docs/285941.htm It is supposedly fixed in MP5, but I've yet to get confirmation on this from anyone out in the field that had experienced the issue MP5 http://seer.entsupport.symantec.com/docs/290387.htm If you are running anything below MP5: and, if you pempersist file, which is in the following directory in a windows environment, Install Path\Program Files\VERITAS\NetBackup\bin\bpsched.d\, gets above about 260K in size, you'll likely start to see odd behavior/errors from the scheduler; not all of which readily appear related to the scheduler. A prime example is the status 41 errors. Good luck. Kent Eagle MTS Infrastructure Engineer II, MCP, MCSE Tech Services / SMSS Visit our website at www.wilmingtontrust.com Investment products are not insured by the FDIC or any other governmental agency, are not deposits of or other obligations of or guaranteed by Wilmington Trust or any other bank or entity, and are subject to risks, including a possible loss of the principal amount invested. This e-mail and any files transmitted with it may contain confidential and/or proprietary information. It is intended solely for the use of the individual or entity who is the intended recipient. Unauthorized use of this information is prohibited. If you have received this in error, please contact the sender by replying to this message and delete this material from any system it may be on. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu Thanks for the info. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4?
Hi Doug, I'm not holding my breath for MP5 either. I'm waiting for my bpdbm binary before I install MP5 and then I don't expect the pempersist issue to be fixed. I also have some binaries to fix nbpem issues but will eed to make sure I can use them in MP5 as I've been told they are not included until MP6!!! I logged a call saying pempersist was broken in MP4 back in Feb this year and they still could not fix it in time ! Pempersist lives in the below dir on UNIX: /usr/openv/netbackup/db/jobs I have not noticed any relation between size and scheduler issues. My pem files have been broken when they are as small as 125K to 700K in size. Currently I am running OK with a 700K pem file. But it's fairly straight forward to see when it's broken. You start to see clients not being backed up, DSUs not being de-staged and as you mentioned the 41s. I wish they would fix it because it's my major gripe with Netbackup right now, otherwise I am actually happy with it now. Cheers, Dom -Original Message- From: Eagle, Kent [mailto:[EMAIL PROTECTED] Sent: Wednesday, 29 August 2007 10:42 PM To: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4? Doug, We have also seen these errors occur in the manner you describe. We traced it to the Pempersist issue, which Veritas support confirmed WAS NOT fixed in MP3 OR MP4 as they originally posted in the MP4 notes. Each maintenance pack claimed the next was going to fix the issue. Sigh... See the following Tech Notes for more info: MP1 http://seer.entsupport.symantec.com/docs/281780.htm MP2 http://seer.entsupport.symantec.com/docs/283180.htm MP3 http://seer.entsupport.symantec.com/docs/285529.htm MP4 http://seer.entsupport.symantec.com/docs/285941.htm It is supposedly fixed in MP5, but I've yet to get confirmation on this from anyone out in the field that had experienced the issue MP5 http://seer.entsupport.symantec.com/docs/290387.htm If you are running anything below MP5: and, if you pempersist file, which is in the following directory in a windows environment, Install Path\Program Files\VERITAS\NetBackup\bin\bpsched.d\, gets above about 260K in size, you'll likely start to see odd behavior/errors from the scheduler; not all of which readily appear related to the scheduler. A prime example is the status 41 errors. Good luck. Kent Eagle MTS Infrastructure Engineer II, MCP, MCSE Tech Services / SMSS Visit our website at www.wilmingtontrust.com Investment products are not insured by the FDIC or any other governmental agency, are not deposits of or other obligations of or guaranteed by Wilmington Trust or any other bank or entity, and are subject to risks, including a possible loss of the principal amount invested. This e-mail and any files transmitted with it may contain confidential and/or proprietary information. It is intended solely for the use of the individual or entity who is the intended recipient. Unauthorized use of this information is prohibited. If you have received this in error, please contact the sender by replying to this message and delete this material from any system it may be on. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
[Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4?
Four or five days ago I began getting connection timed out (41) on my catalog backups to tape, nothing in the environment has changed besides adding some more clients to some polices. However, if a lot of the clients are dead in the polices, I wonder if that could potentially cause this? When a catalog backup is executed though, AFAIK it does not touch the clients? Here is what the error looks like under 6.0MP4: 08/27/2007 15:10:00 - requesting resource MYSTUGROUP 08/27/2007 15:10:00 - requesting resource my-master.fully.fqdn.com.NBU_CLIENT.MAXJOBS.my-master.fully.fqdn.com 08/27/2007 15:10:00 - requesting resource my-master.fully.fqdn.com.NBU_POLICY.MAXJOBS.Catalog 08/27/2007 15:10:00 - granted resource my-master.fully.fqdn.com.NBU_CLIENT.MAXJOBS.my-master.fully.fqdn.com 08/27/2007 15:10:00 - granted resource my-master.fully.fqdn.com.NBU_POLICY.MAXJOBS.Catalog 08/27/2007 15:10:00 - granted resource T1 08/27/2007 15:10:00 - granted resource Drive01 08/27/2007 15:10:00 - granted resource my-media-server.fully.fqdn.com 08/27/2007 15:10:01 - started process bpbrm (pid=2756) 08/27/2007 15:10:01 - connecting 08/27/2007 15:10:01 - connected; connect time: 0:00:00 08/27/2007 15:10:04 - mounted T1 08/27/2007 15:10:04 - positioning T1 to file 16 08/27/2007 15:10:08 - positioned T1; position time: 0:00:04 08/27/2007 15:10:08 - begin writing 08/27/2007 16:27:07 - Error bptm (pid=2825) cannot add fragment to image database, error = network connection timed out 08/27/2007 16:27:34 - end writing; write time: 1:17:26 network connection timed out (41) 08/27/2007 00:25:22 - granted resource my-media-server.fully.fqdn.com 08/27/2007 00:25:27 - started process bpbrm (pid=25775) 08/27/2007 00:25:27 - connecting 08/27/2007 00:25:28 - connected; connect time: 0:00:00 08/27/2007 00:25:31 - mounted T2 08/27/2007 00:25:31 - positioning T2 to file 67 08/27/2007 00:25:34 - positioned T2; position time: 0:00:03 08/27/2007 00:25:34 - begin writing 08/27/2007 01:38:10 - Error bptm (pid=25776) cannot add fragment to image database, error = network connection timed out 08/27/2007 01:43:52 - end writing; write time: 1:18:18 network connection timed out (41) 08/26/2007 00:10:43 - requesting resource MYSTUGROUP 08/26/2007 00:10:43 - requesting resource my-master.fully.fqdn.com.NBU_CLIENT.MAXJOBS.my-master.fully.fqdn.com 08/26/2007 00:10:43 - requesting resource my-master.fully.fqdn.com.NBU_POLICY.MAXJOBS.Catalog 08/26/2007 00:10:58 - granted resource my-master.fully.fqdn.com.NBU_CLIENT.MAXJOBS.my-master.fully.fqdn.com 08/26/2007 00:10:58 - granted resource my-master.fully.fqdn.com.NBU_POLICY.MAXJOBS.Catalog 08/26/2007 00:10:58 - granted resource T3 08/26/2007 00:10:58 - granted resource Drive02 08/26/2007 00:10:58 - granted resource my-media-server.fully.fqdn.com 08/26/2007 00:11:01 - started process bpbrm (pid=10866) 08/26/2007 00:11:01 - connecting 08/26/2007 00:11:02 - connected; connect time: 0:00:00 08/26/2007 00:11:04 - mounting T3 08/26/2007 00:11:44 - mounted T3; mount time: 0:00:40 08/26/2007 00:11:45 - positioning T3 to file 10 08/26/2007 00:13:03 - positioned T3; position time: 0:01:18 I ran a consistency check (bpdbm) and bpexpdate -deassignempty has anyone ever seen this before? ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4?
Try increasing your network timeouts. Since I added 400 day retention to my catalogs and log files (Blame The Auditors) I have had to increase my timeouts at least twice in the last year. I know it sounds funny to need to increase them on the master server when the master server is backing up the catalog but it made a difference. Doug Preston Systems Engineer Land America Tax and Flood Services Phone 626-339-5221 Ext 1104 Email [EMAIL PROTECTED] NOTICE: This electronic mail transmission may constitute a communication that is legally privileged. It is not intended for transmission to, or receipt by, any unauthorized persons. If you have received this electronic mail transmission in error, please delete it from your system without copying it, and notify the sender by reply e-mail, so that our address record can be corrected. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Justin Piszcz Sent: Tuesday, August 28, 2007 7:42 AM To: veritas-bu@mailman.eng.auburn.edu Subject: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4? Four or five days ago I began getting connection timed out (41) on my catalog backups to tape, nothing in the environment has changed besides adding some more clients to some polices. However, if a lot of the clients are dead in the polices, I wonder if that could potentially cause this? When a catalog backup is executed though, AFAIK it does not touch the clients? Here is what the error looks like under 6.0MP4: 08/27/2007 15:10:00 - requesting resource MYSTUGROUP 08/27/2007 15:10:00 - requesting resource my-master.fully.fqdn.com.NBU_CLIENT.MAXJOBS.my-master.fully.fqdn.com 08/27/2007 15:10:00 - requesting resource my-master.fully.fqdn.com.NBU_POLICY.MAXJOBS.Catalog 08/27/2007 15:10:00 - granted resource my-master.fully.fqdn.com.NBU_CLIENT.MAXJOBS.my-master.fully.fqdn.com 08/27/2007 15:10:00 - granted resource my-master.fully.fqdn.com.NBU_POLICY.MAXJOBS.Catalog 08/27/2007 15:10:00 - granted resource T1 08/27/2007 15:10:00 - granted resource Drive01 08/27/2007 15:10:00 - granted resource my-media-server.fully.fqdn.com 08/27/2007 15:10:01 - started process bpbrm (pid=2756) 08/27/2007 15:10:01 - connecting 08/27/2007 15:10:01 - connected; connect time: 0:00:00 08/27/2007 15:10:04 - mounted T1 08/27/2007 15:10:04 - positioning T1 to file 16 08/27/2007 15:10:08 - positioned T1; position time: 0:00:04 08/27/2007 15:10:08 - begin writing 08/27/2007 16:27:07 - Error bptm (pid=2825) cannot add fragment to image database, error = network connection timed out 08/27/2007 16:27:34 - end writing; write time: 1:17:26 network connection timed out (41) 08/27/2007 00:25:22 - granted resource my-media-server.fully.fqdn.com 08/27/2007 00:25:27 - started process bpbrm (pid=25775) 08/27/2007 00:25:27 - connecting 08/27/2007 00:25:28 - connected; connect time: 0:00:00 08/27/2007 00:25:31 - mounted T2 08/27/2007 00:25:31 - positioning T2 to file 67 08/27/2007 00:25:34 - positioned T2; position time: 0:00:03 08/27/2007 00:25:34 - begin writing 08/27/2007 01:38:10 - Error bptm (pid=25776) cannot add fragment to image database, error = network connection timed out 08/27/2007 01:43:52 - end writing; write time: 1:18:18 network connection timed out (41) 08/26/2007 00:10:43 - requesting resource MYSTUGROUP 08/26/2007 00:10:43 - requesting resource my-master.fully.fqdn.com.NBU_CLIENT.MAXJOBS.my-master.fully.fqdn.com 08/26/2007 00:10:43 - requesting resource my-master.fully.fqdn.com.NBU_POLICY.MAXJOBS.Catalog 08/26/2007 00:10:58 - granted resource my-master.fully.fqdn.com.NBU_CLIENT.MAXJOBS.my-master.fully.fqdn.com 08/26/2007 00:10:58 - granted resource my-master.fully.fqdn.com.NBU_POLICY.MAXJOBS.Catalog 08/26/2007 00:10:58 - granted resource T3 08/26/2007 00:10:58 - granted resource Drive02 08/26/2007 00:10:58 - granted resource my-media-server.fully.fqdn.com 08/26/2007 00:11:01 - started process bpbrm (pid=10866) 08/26/2007 00:11:01 - connecting 08/26/2007 00:11:02 - connected; connect time: 0:00:00 08/26/2007 00:11:04 - mounting T3 08/26/2007 00:11:44 - mounted T3; mount time: 0:00:40 08/26/2007 00:11:45 - positioning T3 to file 10 08/26/2007 00:13:03 - positioned T3; position time: 0:01:18 I ran a consistency check (bpdbm) and bpexpdate -deassignempty has anyone ever seen this before? ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4?
On Tue, 28 Aug 2007, Preston, Douglas L wrote: Try increasing your network timeouts. Since I added 400 day retention to my catalogs and log files (Blame The Auditors) I have had to increase my timeouts at least twice in the last year. I know it sounds funny to need to increase them on the master server when the master server is backing up the catalog but it made a difference. What specific tunables did you change? Thanks, Justin. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Re: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4?
CLIENT_READ_TIMEOUT to the /usr/openv/netbackup/bp.conf file and set it to increase the timeout interval. Make sure all recommended NetBackup patches are installed. Check the Symantec support Web site for current patch information. (Go to www.support.veritas.com. Then select NetBackup followed by files and updates.) Run the NetBackup Configuration Validation Utility (NCVU) for the associated NetBackup nodes. Note the pack checks in section two. Add the CLIENT_READ_TIMEOUT values to the master server, media server, and client when a NetBackup database extension product is installed. The values should all be the same for each server. The value set is dependent on the size of the database being backed up. See the NetBackup Administrator's Guide, Volume II, for more information on CLIENT_READ_TIMEOUT. Make sure enhanced authentication is configured correctly. For example, the following may result in status code 41: host A is configured to use enhanced authentication with host B, but host B is not configured to use enhanced authentication with host A. In this case, connections from host B to host A are likely to fail with status code 41. Connections from host A to B are likely to fail with authentication errors (status code 160). Doug Preston Systems Engineer Land America Tax and Flood Services Phone 626-339-5221 Ext 1104 Email [EMAIL PROTECTED] NOTICE: This electronic mail transmission may constitute a communication that is legally privileged. It is not intended for transmission to, or receipt by, any unauthorized persons. If you have received this electronic mail transmission in error, please delete it from your system without copying it, and notify the sender by reply e-mail, so that our address record can be corrected. -Original Message- From: Justin Piszcz [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 28, 2007 9:08 AM To: Preston, Douglas L Cc: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Error 41 on Catalog Backup using NBU 6.0MP4? On Tue, 28 Aug 2007, Preston, Douglas L wrote: Try increasing your network timeouts. Since I added 400 day retention to my catalogs and log files (Blame The Auditors) I have had to increase my timeouts at least twice in the last year. I know it sounds funny to need to increase them on the master server when the master server is backing up the catalog but it made a difference. What specific tunables did you change? Thanks, Justin. ___ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
