Re: [cross-project-issues-dev] Sorry, entering a bug into the product JDT has been disabled.,

2022-04-20 Thread Matthew Ward
Ok I"ll be in tomorrow On Tue, Apr 19, 2022 at 9:19 AM Denis Roy wrote: > Ed, > > I'm very sorry that you choose to see this as a vandalism. Change is > always difficult to manage; yet if we refusing to accept it, we risk being > left behind while our surroundings evolve around us. We've been

[cross-project-issues-dev] Emergency Outage - OKD cluster

2021-06-23 Thread Matthew Ward
Hello Everyone, We are experiencing an outage in our OKD cluster due to failed network change. Currently the webmaster team is working to restore service, but we may need to restart the cluster. -Matt. ___ cross-project-issues-dev mailing list

[cross-project-issues-dev] Possible service degredation: April 22 00:00-06:00AM EDT

2021-04-19 Thread Matthew Ward
Hi Everyone, We've just been informed that our Datacenter is being forced to migrate one of it's fibre uplinks due to some construction in the area. The risk of an outage is low as there are redundant links,, but they estimate there could be up to 45m of actual downtime. While this work is

[cross-project-issues-dev] Downloads/Archives file permission change.

2020-05-27 Thread Matthew Ward
Hi Everyone, As part of trying to keep things tidy in the filesystem we've removed all the old extended ACL entries on files located on the download/archives servers. These were used to help us bypass limits on group membership imposed by NFS. What does this mean for you? Hopefully nothing,

Re: [cross-project-issues-dev] Missing emails from Bugzilla today

2020-05-15 Thread Matthew Ward
This was a side effect of work to archive the PDT bugzilla product. In order to prevent 'spamming' people with a few thousand(6096 to be specific) bug notices I had disabled mail delivery for a few minutes. -Matt. On Thu, May 14, 2020 at 5:06 PM Jonah Graham wrote: > Hello folks, > > AFAICT a

Re: [cross-project-issues-dev] ci.eclipse.org/gef/ is down

2020-05-06 Thread Matthew Ward
I've restarted the instance and it's back online. -Matt On Wed, May 6, 2020 at 9:46 AM Victor V. Rubezhny wrote: > Hello, > > https://ci.eclipse.org/gef is down (504 Gateway Timeout) which causes > some Gerrit builds to fail. > > Could anybody help with this issue? > > > Thanks in advance, > >

Re: [cross-project-issues-dev] Gerrit errors?

2020-04-30 Thread Matthew Ward
Actually I'm just updating 561802 with the details from this. In the meantime Gerrit has been restarted to hopefully minimize any other issues. -Matt. On Thu, Apr 30, 2020 at 2:27 PM Jonah Graham wrote: > The webmasters often see stuff

Re: [cross-project-issues-dev] Cloning Via git://git.eclipse.org/gitroot Fails

2020-03-03 Thread Matthew Ward
Hi Ed, The core of these failures is probably the same as the issue that impacted the mail service, and should now be resolved. -Matt. On Tue, Mar 3, 2020 at 10:23 AM Ed Merks wrote: > FYI, > > All my build jobs are failing the for same reason as reported by > Matthias Sohn: > >

Re: [cross-project-issues-dev] Bug 560709 - Gerrit not serving to jenkins and not sending emails

2020-03-03 Thread Matthew Ward
Hi All, I've taken a look and the issue seems to have been caused by an improperly escaped permissions change that altered the root filesystem on the node that provides our primary mail service. We've fixed the permissions, and mail has started moving again. There shouldn't be any significant

[cross-project-issues-dev] Gerrit service restart

2020-02-24 Thread Matthew Ward
Hello everyone, The Gerrit process is currently reporting dead threads which is impacting new account creation and other services that require them. I will be restarting the service in 20 minutes. I expect it to be offline for about 5 minutes while it shuts down and restarts. -Matt.

Re: [cross-project-issues-dev] Issues with git/gerrit and sometimes bugzilla

2020-02-07 Thread Matthew Ward
One of our backend storage nodes fell over which has impacted service availability. We've cut over to our standby and are restarting services as required to bring everything back up. -Matt. On Fri, Feb 7, 2020 at 2:03 PM Thomas Watson wrote: > I've been getting 502 errors access the eclipse

Re: [cross-project-issues-dev] End of an Era: shell access.

2019-09-03 Thread Matthew Ward
this should remove the old content: > r$ rm -rf /home/data/httpd/download.eclipse.org/egit/github/updates/* > The issue is that the restricted shell doesn't do any pattern expansion/globbing of things like * or ? on the commands it supports. I'll talk to the other members of the Webmaster team

Re: [cross-project-issues-dev] End of an Era: shell access.

2019-08-29 Thread Matthew Ward
at 2:52 AM Eike Stepper wrote: > Am 27.08.2019 um 16:44 schrieb Matthew Ward: > > Hi Ed, > > > > The restricted shell was originally created with the goal of providing > committers a way to interact with the > > downloads/archive filesystems for releng activities,

Re: [cross-project-issues-dev] End of an Era: shell access.

2019-08-27 Thread Matthew Ward
gt; restricted shell. > > I should point out that this is quite different from the impression that > was presented earlier in this process, so I'm disappointed in how this has > been presented and handled. It feels to me like a process of decree where > there is no recourse: >

Re: [cross-project-issues-dev] End of an Era: shell access.

2019-08-27 Thread Matthew Ward
ed could become unrestricted... > > On 26.08.2019 15:35, Matthew Ward wrote: > > Hi David, > > Thanks for the questions. > > Users with the restricted shell will have the same home directories that > they do currently, which will remain the place for authorized keys. You

Re: [cross-project-issues-dev] End of an Era: shell access.

2019-08-26 Thread Matthew Ward
, Aug 24, 2019 at 3:23 PM David Williams wrote: > On 8/23/19 14:24, Matthew Ward wrote: > > Hi Everyone, > > I just wanted to follow up with a reminder that on August 28th we will > be moving committers that have an actual shell on Eclipse.org to our > restricted shell. >

[cross-project-issues-dev] End of an Era: shell access.

2019-08-23 Thread Matthew Ward
Hi Everyone, I just wanted to follow up with a reminder that on August 28th we will be moving committers that have an actual shell on Eclipse.org to our restricted shell. I'd like to thank both Donat and Etienne on the Buildship RelEng team who volunteered to test this change, and helped me

[cross-project-issues-dev] Mac signing: service outage

2019-08-23 Thread Matthew Ward
Hello, I just wanted to let everyone know that we have experienced a failure in the mac signing service. We are working to restore it, but have had to reach out to Apple for assistance. At this time I expect us to have things operational again by EOD Monday Aug 26th. Should we be able to

Re: [cross-project-issues-dev] hipp 3 down ?

2019-06-11 Thread Matthew Ward
I've restarted hipp3 and the Jenkins instances are coming back. On Tue, Jun 11, 2019, 3:55 AM Matthias Sohn wrote: > On Tue, Jun 11, 2019 at 9:52 AM Matthias Sohn > wrote: > >> I can't reach the JGit Jenkins [1] running on hipp3 and the infra monitor >> [2] says it is offline. >> Can you

Re: [cross-project-issues-dev] Moving to Gerrit

2018-06-05 Thread Matthew Ward
Ultimately it's totally arbitrary. When I decided to do this[1] , I needed to pick a date. After Ed pointed out that +0 projects are RC3 this week I offered to move it, but since nobody responded to that I took it to mean I could go forward. However I'm willing to hold off if this list is

Re: [cross-project-issues-dev] LDAP error when logging into bugzilla

2018-04-27 Thread Matthew Ward
We're currently working on this, it looks like HTTP connections have piled up on one node swamping the LDAP server -Matt. On Fri, Apr 27, 2018 at 9:28 AM, Eric Williams wrote: > Hello, > > I am having trouble logging into the Eclipse.org bugzilla or gerrit. When > logging

Re: [cross-project-issues-dev] HIPPs residing on hipp3 not responding

2017-02-15 Thread Matthew Ward(Bt. Kt. SSA.)
Unfortunately this was not a planned outage. Our attempts to revive the server remotely have been unsuccessful so one of the Webaster team is on their way to the co-lo facility to intervene manually. On 02/15/2017 09:54 AM, Roland Grunberg wrote: > >From https://hudson.eclipse.org/, none of the

Re: [cross-project-issues-dev] Bugzilla down?

2016-12-06 Thread Matthew Ward(Bt. Kt. SSA.)
I've restarted bugs-vm2, it was taken offline by an error in the hosts NIC firmware. And download3+4 are not currently in service while we test them, so they should have no impact on download performance. -Matt. On 12/06/2016 06:01 AM, Markus Keller wrote: > Yes, Bugzilla is down. Not for

Re: [cross-project-issues-dev] Push to Gerrit fails with Permission denied during unpacking

2016-11-17 Thread Matthew Ward(Bt. Kt. SSA.)
This sounds like bug 507374, but I'll have to dig into what is going on. For now you should be able to commit without issue. -Matt. On 11/17/2016 04:30 AM, Wenz, Michael wrote: > > Hi, > > > > when I try to push a new change in the Graphiti Git to Gerrit I get > this error from the Git

Re: [cross-project-issues-dev] Are restrictions on Bugzilla worse thanspam?

2016-11-14 Thread Matthew Ward(Bt. Kt. SSA.)
> With all due respect (and you know I do :) ), I have counter > examples on how effective it is, at least lately, where a CDT > committer still had to wait too long to get his Wiki access > granted as we were putting things together for the Neon.2 release. > I would argue that

Re: [cross-project-issues-dev] Bugzilla is down?

2016-11-14 Thread Matthew Ward(Bt. Kt. SSA.)
This should now be resolved. One of the nodes that backs bugzilla threw an exception for it's backend nic which prevented it from talking to the database network, and since it's primary interface was still working it wasn't being removed by the load balancer. We've restarted the node and it's

Re: [cross-project-issues-dev] Bugzilla is down?

2016-11-14 Thread Matthew Ward(Bt. Kt. SSA.)
I'm working on it -Matt. On 11/14/2016 09:15 AM, Andrey Loskutov wrote: > Let say it is now *a bit* better but doesn't solve my problem: > > Accessing https://bugs.eclipse.org returns this: > > > Internal Server Error > > The server encountered an internal error or misconfiguration and was >

Re: [cross-project-issues-dev] Upgrading HIPPs to SLES12 (Part 2 - reloaded)

2016-10-24 Thread Matthew Ward(Bt. Kt. SSA.)
We are currently restoring the jobs data, once that's finished(~2hours) you should be able to restart your instance and run builds as normal. -Matt. On 24/10/16 01:01 PM, Tom Schindl wrote: > Any idea when the e(fx)clipse one is up again? It's now offline for > 3 > days and I really want to run

Re: [cross-project-issues-dev] Bugzilla down ?

2016-10-21 Thread Matthew Ward(Bt. Kt. SSA.)
Hi Folks, Early this morning we experienced a complete outage of our primary database server. Denis and I are investigating but as of right now the server is back online and I'm working on making sure the slave server is replicating correctly. -Matt.

[cross-project-issues-dev] Maven downtime.

2016-07-05 Thread Matthew Ward(Bt. Kt. SSA.)
Hi Folks, Just a reminder that I will be shutting down repo.eclipse.org in order to move it to a new host this afternoon starting at 2pm. I expect the outage to last for a couple of hours. -Matt. ___ cross-project-issues-dev mailing list

Re: [cross-project-issues-dev] Wiki and PMI down ?

2016-07-04 Thread Matthew Ward(Bt. Kt. SSA.)
Both services should now be functional again. -Matt. On 07/04/2016 10:50 AM, Daniel Megert wrote: > >We've fixed both of those. > > Not really: https://bugs.eclipse.org/497212 > > Dani > > > > From:Denis Roy > To:cross-project-issues-dev@eclipse.org >

Re: [cross-project-issues-dev] Bugzilla + Gerrit + ? down

2016-06-16 Thread Matthew Ward(Bt. Kt. SSA.)
We've got things going on the master and the slave should be catching up. -Matt. On 06/16/2016 09:04 AM, Matthias Sohn wrote: > Gerrit seems to work again ___ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org To change your

Re: [cross-project-issues-dev] Bugzilla + Gerrit + ? down

2016-06-16 Thread Matthew Ward(Bt. Kt. SSA.)
Our master database server has run out of disk space and the slave process is stalled. I'm working on getting things back up. -Matt. On 06/16/2016 08:05 AM, Patrik Suzzi wrote: > Same for me, in three sites. > > https://git.eclipse.org/r/ --> Git/Gerrit > > Server Error > >

Re: [cross-project-issues-dev] Known outage?

2015-11-25 Thread Matthew Ward(Bt. Kt. SSA.)
I think some of these were caused by a new machine added to the proxy pool today. I think it's nailed down now. -Matt On 11/25/2015 02:36 PM, Marc-André Laperle wrote: > There seems to be some intermittent issues connecting to several > servers, like repo.maven.apache.org and

Re: [cross-project-issues-dev] Hudson fails to access http://download.eclipse.org/eclipse/updates/4.4-M-builds

2015-02-01 Thread Matthew Ward(Bt. Kt. SSA.)
It looks like our ssh attack protection blocked part of our internal network because of some bad longterm stats. I've cleared the block, and the data behind it, and now EPP is building again. -Matt. On 02/01/2015 07:25 AM, Markus Knauer wrote: EPP build cannot access Git:

Re: [cross-project-issues-dev] build problems

2013-04-19 Thread Matthew Ward(Bt. Kt. SSA.)
I'm working to restore the ACLs. -Matt. We also have permission issues with our build. It seems that the ACLs permissions on /shared are all lost (in our case). -Gunnar ___ cross-project-issues-dev mailing list

Re: [cross-project-issues-dev] Server switch

2013-04-18 Thread Matthew Ward(Bt. Kt. SSA.)
Hi Carl, Yes they are. Sorry I should have made that more clear and I didn't. -Matt. On 04/18/2013 03:46 PM, Carl Anderson wrote: Matt, Yes, one concern- does the same server that provides