bd808 added a comment.
@Smalyshev please make a separate task for the RAM quota bump so that we can keep track of it and drop it back down after you build the new VM and shutdown an old one. There is a handy link on #cloud-vps-quota-requests that will give you a template task to fill in.TASK DETAIL
Smalyshev added a comment.
I see it but cannot use it because it goes over the RAM quota. Can we have it increased a little for wikidata-query?TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Andrew, Smalyshev
Andrew added a comment.
ok, flavor added. I'm a bit nervous about how the scheduler will handle this, so please ping me after your instance is created but before you get attached to it so I can make sure it landed someplace reasonable.TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFE
bd808 added a comment.
The project is wikidata-query.TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Andrew, bd808Cc: daniel, Lydia_Pintscher, bd808, chasemp, Gehel, Aklapper, Smalyshev, IMBLUESKY, Lucas_Werk
Andrew added a comment.
I don't see a project named 'WDQS' -- can you clarify what actual VPS project we're talking about here? I see a few things it could be...TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To
chasemp added a comment.
+1'd to grant access to the 300G flavor via meetingTASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chasempCc: daniel, Lydia_Pintscher, bd808, chasemp, Gehel, Aklapper, Smalyshev, Luca
chasemp added a comment.
I'm making a note to discuss it today in our meeting ;)TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chasempCc: daniel, Lydia_Pintscher, bd808, chasemp, Gehel, Aklapper, Smalyshev,
Gehel added a comment.
That would be great! If you grant us access to that flavor with 300GB disk, we don't event need a quota extension. We have one instance (wdq-beta) that needs to be deleted after some checks, so we should be all good.TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PR
chasemp added a comment.
@Gehel
We could bump up the quota temporarily to allow rebuild of those instances with a flavor that has a larger disk. We have a flavor that has 300G of disk already we have granted selectively elsewhere so that would work out. I believe this is a 20G root partition and
Gehel added a comment.
@Smalyshev I'm wondering what the performance bottleneck is. I suppose it is IO, but with the current state of wdqs-beta, it is hard to confirm. We might be able to get better performances from the same resources by tuning things a bit. Reducing the memory allocated to the JV
Smalyshev added a comment.
Yes, the main problem is the diskspace. Though the performance is kinda an issue too since we can't keep up with full update stream on VMs. But running out of space is the most immediate one.TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phab
Gehel added a comment.
To summarize an IRC discussion with @chasemp:
The specs above (8 CPU, 16Gb RAM, 500Gb disk) are the same as the current wdq-deploy and wdqs-beta. So the increase is only disk
Disk is sadly the most contentious resource
Expanding our cloud means new hardware, which implies t
Smalyshev added a comment.
@chasemp two should be enough - one for wdq-beta and one for wdq-deploy.TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: daniel, Lydia_Pintscher, bd808, chasemp, Gehel,
chasemp added a comment.
How many VMs at the spec?TASK DETAILhttps://phabricator.wikimedia.org/T169133EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chasempCc: daniel, Lydia_Pintscher, bd808, chasemp, Gehel, Aklapper, Smalyshev, GoranSMilovanovic, QZanden, EB
Gehel added a comment.
Doing some more guess work:
IO performance on a test instance of WDQS is mostly limited by writes, which should be similar to what we have in production. Grafana shows writes between 500 and 1000 IOPS.
Looking at the GC logs on wdqs-beta (for the last 24 days), it looks lik
bd808 added a comment.
In T169133#3389187, @Smalyshev wrote:
I do not insist on any particular solution for this one, but I think we need to start thinking about how we can run real-hardware non-production setups, or cover such requirements in some other way.
It is certainly a topic worth discus
Smalyshev added a comment.
I think this is not the last time we're hearing about such setup. With more and more data being processed and us starting to use algorithms and tools that require significant computing power (ML, graph DBs, etc.) we'd have more use cases that need storage beyond 160G and
17 matches
Mail list logo
