Re: [Bf-committers] Blender server upgrades
Hi, It seems I found a few things of concern while digging relating to file permissions in the file system: https://download.blender.org/ftp/dan/admin/phab_permissions/Screenshot_2020-01-20_15-41-51.png Apparently there are some root owned files and directories, one of which was that object file, as well as it's directories. Likely the permissions are blocking things, and should be reset to www:www. That said, it's possible that some cron based tool in the system could also be generating root owned files, but anything invoked from the webserver would most certainly need to be owned by www. If root files are an issue, we should be able to solve via a umask in the system. That said, we may need to go through and just verify that there were no repo corruptions from being unable to pack files, but I am not sure that is a problem with these particular files in git. A quick check on file timestamps suggests that the problem may have actually started around January 9th, or earlier, which is before the upgrades happened. Most likely the two had nothing to do with each other, which is good to know: https://download.blender.org/ftp/dan/admin/phab_permissions/Screenshot_2020-01-20_15-49-50.png https://download.blender.org/ftp/dan/admin/phab_permissions/Screenshot_2020-01-20_16-06-54.png Anyway, I will probably avoid touching this personally, pending approval of Sergey and folks, but we at least know where the problem lies now. Cheers, Dan McGrath On Mon, Jan 20, 2020 at 3:15 PM Dan McGrath wrote: > Hi, > > Just another heads up that today it was reported that there was still some > delay on the time between when the commits happen, and when they get > processed and finally sent out. > > It was said that the CVS mailing list mails that happen when the commit is > received by git were happening, so it is most likely a delay in the > polling, and/or processing of the commit itself from the repo. > > I can take a look into a few more things shortly here, but Sergey won't > have a look until at least tomorrow. In the meantime, I have restarted the > PHD scripts that process commits, but expect 10-20 minute delays in the > time between your commits and the time that Phabricator sees them. > > Sorry for the inconvenience! > > > Cheers, > > Dan McGrath > > On Sat, Jan 18, 2020 at 2:42 PM Dan McGrath > wrote: > >> Hi, >> >> We were looking into it on the blender chat, and nobody touched anything >> then suddenly a few mails arrived. Checking the mail logs on the >> Phabricator system shows they didn't even queue until 10 minutes after the >> commit, so maybe something a little lagged in the system. At least know it >> wasn't the mail system, just based on the logs. >> >> I can only guess that git or svn was under some traffic or phab was busy >> on processing or something for 10 minutes? Anywho, seems fine now, but >> maybe Sergey can dig deeper. PHD's were/are running fine afaict. >> >> >> Cheers, >> >> Dan >> >> On Sat, Jan 18, 2020 at 2:12 PM Ray Molenkamp wrote: >> >>> not sure what changed but phab is either no longer or real slow picking >>> up on git commits >>> >>> --Ray >>> >>> On 2020-01-17 1:01 p.m., Dan McGrath wrote: >>> > Hi, >>> > >>> > Just a heads up that I will be attempting to upgrade some of the >>> FreeBSD >>> > servers and packages tonight, so there will be reboots. >>> > >>> > So, go out, enjoy your Friday night, and get off the internet so I can >>> poke >>> > at things :) o/ >>> > >>> > >>> > Cheers, >>> > >>> > Dan McGrath >>> > ___ >>> > Bf-committers mailing list >>> > Bf-committers@blender.org >>> > https://lists.blender.org/mailman/listinfo/bf-committers >>> ___ >>> Bf-committers mailing list >>> Bf-committers@blender.org >>> https://lists.blender.org/mailman/listinfo/bf-committers >>> >> ___ Bf-committers mailing list Bf-committers@blender.org https://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Blender server upgrades
Hi, Just another heads up that today it was reported that there was still some delay on the time between when the commits happen, and when they get processed and finally sent out. It was said that the CVS mailing list mails that happen when the commit is received by git were happening, so it is most likely a delay in the polling, and/or processing of the commit itself from the repo. I can take a look into a few more things shortly here, but Sergey won't have a look until at least tomorrow. In the meantime, I have restarted the PHD scripts that process commits, but expect 10-20 minute delays in the time between your commits and the time that Phabricator sees them. Sorry for the inconvenience! Cheers, Dan McGrath On Sat, Jan 18, 2020 at 2:42 PM Dan McGrath wrote: > Hi, > > We were looking into it on the blender chat, and nobody touched anything > then suddenly a few mails arrived. Checking the mail logs on the > Phabricator system shows they didn't even queue until 10 minutes after the > commit, so maybe something a little lagged in the system. At least know it > wasn't the mail system, just based on the logs. > > I can only guess that git or svn was under some traffic or phab was busy > on processing or something for 10 minutes? Anywho, seems fine now, but > maybe Sergey can dig deeper. PHD's were/are running fine afaict. > > > Cheers, > > Dan > > On Sat, Jan 18, 2020 at 2:12 PM Ray Molenkamp wrote: > >> not sure what changed but phab is either no longer or real slow picking >> up on git commits >> >> --Ray >> >> On 2020-01-17 1:01 p.m., Dan McGrath wrote: >> > Hi, >> > >> > Just a heads up that I will be attempting to upgrade some of the FreeBSD >> > servers and packages tonight, so there will be reboots. >> > >> > So, go out, enjoy your Friday night, and get off the internet so I can >> poke >> > at things :) o/ >> > >> > >> > Cheers, >> > >> > Dan McGrath >> > ___ >> > Bf-committers mailing list >> > Bf-committers@blender.org >> > https://lists.blender.org/mailman/listinfo/bf-committers >> ___ >> Bf-committers mailing list >> Bf-committers@blender.org >> https://lists.blender.org/mailman/listinfo/bf-committers >> > ___ Bf-committers mailing list Bf-committers@blender.org https://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Blender server upgrades
Hi, We were looking into it on the blender chat, and nobody touched anything then suddenly a few mails arrived. Checking the mail logs on the Phabricator system shows they didn't even queue until 10 minutes after the commit, so maybe something a little lagged in the system. At least know it wasn't the mail system, just based on the logs. I can only guess that git or svn was under some traffic or phab was busy on processing or something for 10 minutes? Anywho, seems fine now, but maybe Sergey can dig deeper. PHD's were/are running fine afaict. Cheers, Dan On Sat, Jan 18, 2020 at 2:12 PM Ray Molenkamp wrote: > not sure what changed but phab is either no longer or real slow picking up > on git commits > > --Ray > > On 2020-01-17 1:01 p.m., Dan McGrath wrote: > > Hi, > > > > Just a heads up that I will be attempting to upgrade some of the FreeBSD > > servers and packages tonight, so there will be reboots. > > > > So, go out, enjoy your Friday night, and get off the internet so I can > poke > > at things :) o/ > > > > > > Cheers, > > > > Dan McGrath > > ___ > > Bf-committers mailing list > > Bf-committers@blender.org > > https://lists.blender.org/mailman/listinfo/bf-committers > ___ > Bf-committers mailing list > Bf-committers@blender.org > https://lists.blender.org/mailman/listinfo/bf-committers > ___ Bf-committers mailing list Bf-committers@blender.org https://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Blender server upgrades
not sure what changed but phab is either no longer or real slow picking up on git commits --Ray On 2020-01-17 1:01 p.m., Dan McGrath wrote: > Hi, > > Just a heads up that I will be attempting to upgrade some of the FreeBSD > servers and packages tonight, so there will be reboots. > > So, go out, enjoy your Friday night, and get off the internet so I can poke > at things :) o/ > > > Cheers, > > Dan McGrath > ___ > Bf-committers mailing list > Bf-committers@blender.org > https://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org https://lists.blender.org/mailman/listinfo/bf-committers
[Bf-committers] Blender server upgrades
Hi, Just a heads up that I will be attempting to upgrade some of the FreeBSD servers and packages tonight, so there will be reboots. So, go out, enjoy your Friday night, and get off the internet so I can poke at things :) o/ Cheers, Dan McGrath ___ Bf-committers mailing list Bf-committers@blender.org https://lists.blender.org/mailman/listinfo/bf-committers
