Re: Possible Performance Regression with Mesa
Hi Joao, On Fri, 26 Apr 2024 at 08:42, Joao Paulo Silva Goncalves wrote: > On Thu, Apr 25, 2024 at 9:08 AM Lucas Stach wrote: > > I can reproduce the issue, but sadly there is no simple fix for this, > > as it's a bad interaction between some of the new features. > > At the core of the issue is the dmabuf-feedback support with the chain > > of events being as follows: > > > 1. weston switches to the scanout tranche, as it would like to put the > > surface on a plane > > 2. the client reallocates as linear but does so on the render node > > 3. weston still isn't able to put the buffer on the plane, as it's > > still scanout incompatible due to being non-contig, so needs to fall > > back to rendering > > 4. now we are stuck at a linear buffer being used for rendering, which > > is very non-optimal > > > I'll look into improving this, but can make no commitments as to when > > I'll be able to get around to this. > > Seem to be tricky. > If you want, we at least can help you test it. Just reach out. > We also saw similar behaviour on more modern hardware, like the iMX8MM. > I will do a bit more testing on the iMX8MM and also some on the iMX8MP to > geather more data and > I am thinking in also opening an issue on the gitlab of Mesa, for better > tracking. What do you think? One thing you can try is to edit weston/libweston/backend-drm/state-propose.c and, inside dmabuf_feedback_maybe_update(), prevent action_needed from ever being set to ACTION_NEEDED_ADD_SCANOUT_TRANCHE. It would be interesting to know if this restores full performance. Cheers, Daniel
Re: Possible Performance Regression with Mesa
On Thu, 25 Apr 2024 at 13:08, Lucas Stach wrote: > I can reproduce the issue, but sadly there is no simple fix for this, > as it's a bad interaction between some of the new features. > At the core of the issue is the dmabuf-feedback support with the chain > of events being as follows: > > 1. weston switches to the scanout tranche, as it would like to put the > surface on a plane > 2. the client reallocates as linear but does so on the render node > 3. weston still isn't able to put the buffer on the plane, as it's > still scanout incompatible due to being non-contig, so needs to fall > back to rendering > 4. now we are stuck at a linear buffer being used for rendering, which > is very non-optimal Oh man, sorry about that, that shouldn't happen. As long as drmModeAddFB2 is failing, we should be marking the buffer as non-importable, and then hinting the client back towards tiled. That being said, yeah, having the client render to linear and skip composition is definitely going to be better! Cheers, Daniel
Re: Future direction of the Mesa Vulkan runtime (or "should we build a new gallium?")
On Fri, 26 Jan 2024 at 00:22, Faith Ekstrand wrote: > On Thu, Jan 25, 2024 at 5:06 PM Gert Wollny wrote: >> I think with Venus we are more interested in using utility libraries on >> an as-needed basis. Here, most of the time the Vulkan commands are just >> serialized according to the Venus protocol and this is then passed to >> the host because usually it wouldn't make sense to let the guest >> translate the Vulkan commands to something different (e.g. something >> that is commonly used in a runtime), only to then re-encode this in the >> Venus driver to satisfy the host Vulkan driver - just think Spir-V: >> why would we want to have NIR only to then re-encode it to Spir-V? > > I think Venus is an entirely different class of driver. It's not even really > a driver. It's more of a Vulkan layer that has a VM boundary in the middle. > It's attempting to be as thin of a Vulkan -> Vulkan pass-through as possible. > As such, it doesn't use most of the shared stuff anyway. It uses the dispatch > framework and that's really about it. As long as that code stays in-tree > roughly as-is, I think Venus will be fine. The eternal response: you forgot WSI! Cheers, Daniel
Re: wayland: wrong tiling with wl_drm on Vivante GC2000
Hi Christian, On Thu, 30 Mar 2023 at 20:15, Christian Gudrian wrote: > We're running a custom Wayland compositor based on Qt Wayland on an i.MX6 > Quad system with a Vivante GC2000 GPU using the Mesa Gallium driver. While > the compositor itself displays correctly, client buffers are displayed with > wrong tiling. > > So far I've found out, that the client's __DRIimage is created by > etna_resource_create with a tiled layout. This image is later passed to > create_wl_buffer which ignores the tiling information and passes the > image's FD to wl_drm_create_prime_buffer. > > If I patch etna_resource_create to always use a linear layout the client's > buffer is displayed correctly. That's certainly not how to do it correctly. > Is there a way to ensure a linear layout for this use case? > You need to force QtWayland to use the wp-linux-dmabuf-v1 platform integration for the compositor. All the others do this automatically but Qt requires an environment variable for this. Cheers, Daniel
Re: [Mesa-dev] Workflow Proposal
On Wed, 13 Oct 2021 at 20:13, Jordan Justen wrote: > Alyssa Rosenzweig writes: > >> Upstream should do what's best for upstream, not for Intel's "unique" > >> management. > >> > >>Not sure how from Emma explaining how Rb tags were used by Intel > >>management it came the conclusion that it were used in that way only by > >>Intel management. Spoiler: it is not. > > > > Sorry, I'll make that point more emphatic. > > > > Upstream must do what's best for upstream without zero regard for the > > whims of management. Doubly so for bad management. > > If the r-b process ever had any notice from any company's management, I > haven't seen it. (Actually, I think most management would rather have > the short sighted view of skipping code review to more quickly merge > patches.) In terms of who to "track down", that is also a tenuous > connection. All of the above is true but also totally irrelevant to the actual discussion. When R-b as a metric came up at the time of the first switch, I wrote a really trivial Python script which used the GitLab API to scrape MR discussions and pull 'Reviewed-by: ...' comments out and print a leaderboard for number of reviewed MRs over the past calendar month. Adapting that to look at approvals rather than comments would cut it down to about 10 LoC. Whether it's Reviewed-by in the trailer or an approval, both are explicitly designed to be machine readable, which means it's trivial to turn it into a metric if you want to. Whether or not that's a good idea is the problem of whoever wields it. Cheers, Daniel
Re: [Mesa-dev] Workflow Proposal
On Thu, 7 Oct 2021 at 09:38, Eero Tamminen wrote: > This sounds horrible from the point of view of trying to track down > somebody who knows about what's & why's of some old commit that is later > on found to cause issues... But why would your first point of call not be to go back to the review discussion and look at the context and what was said at the time? Then when you do that, you can see not only what happened, but also who was involved and saying what at the time. Cheers, Daniel
Re: [Mesa-dev] [PATCH] dma-buf: Document dma-buf implicit fencing/resv fencing rules
Hi, On Wed, 23 Jun 2021 at 17:20, Daniel Vetter wrote: > +* > +* IMPLICIT SYNCHRONIZATION RULES: > +* > +* Drivers which support implicit synchronization of buffer access as > +* e.g. exposed in `Implicit Fence Poll Support`_ should follow the > +* below rules. 'Should' ... ? Must. > +* - Drivers should add a shared fence through > +* dma_resv_add_shared_fence() for anything the userspace API > +* considers a read access. This highly depends upon the API and > +* window system: E.g. OpenGL is generally implicitly synchronized > on > +* Linux, but explicitly synchronized on Android. Whereas Vulkan is > +* generally explicitly synchronized for everything, and window > system > +* buffers have explicit API calls (which then need to make sure the > +* implicit fences store here in @resv are updated correctly). > +* > +* - [...] Mmm, I think this is all right, but it could be worded much more clearly. Right now it's a bunch of points all smashed into one, and there's a lot of room for misinterpretation. Here's a strawman, starting with most basic and restrictive, working through to when you're allowed to wriggle your way out: Rule 1: Drivers must add a shared fence through dma_resv_add_shared_fence() for any read accesses against that buffer. This appends a fence to the shared array, ensuring that any future non-read access will be synchronised against this operation to only begin after it has completed. Rule 2: Drivers must add an exclusive fence through dma_resv_add_excl_fence() for any write accesses against that buffer. This replaces the exclusive fence with the new operation, ensuring that all future access will be synchronised against this operation to only begin after it has completed. Rule 3: Drivers must synchronise all accesses to buffers against existing implicit fences. Read accesses must synchronise against the exclusive fence (read-after-write), and write accesses must synchronise against both the exclusive (write-after-write) and shared (write-after-read) fences. Note 1: Users like OpenGL and window systems on non-Android userspace are generally implicitly synchronised. An implicitly-synchronised userspace is unaware of fences from prior operations, so the kernel mediates scheduling to create the illusion that GPU work is FIFO. For example, an application will flush and schedule GPU write work to render its image, then immediately tell the window system to display that image; the window system may immediately flush and schedule GPU read work to display that image, with neither waiting for the write to have completed. The kernel provides coherence by synchronising the read access against the write fence in the exclusive slot, so that the image displayed is correct. Note 2: Users like Vulkan and Android window system are generally explicitly synchronised. An explicitly-synchronised userspace is responsible for tracking its own read and write access and providing the kernel with synchronisation barriers. For instance, a Vulkan application rendering to a buffer and subsequently using it as a read texture, must annotate the read operation with a read-after-write synchronisation barrier. Note 3: Implicit and explicit userspace can coexist. For instance, an explicitly-synchronised Vulkan application may be running as a client of an implicitly-synchronised window system which uses OpenGL for composition; an implicitly-synchronised OpenGL application may be running as a client of a window system which uses Vulkan for composition. Note 4: Some subsystems, for example V4L2, do not pipeline operations, and instead only return to userspace when the scheduled work against a buffer has fully retired. Exemption 1: Fully self-coherent userspace may skip implicit synchronisation barriers. For instance, accesses between two Vulkan-internal buffers allocated by a single application do not need to synchronise against each other's implicit fences, as the client is responsible for explicitly providing barriers for access. A self-contained OpenGL userspace also has no need to implicitly synchronise its access if the driver instead tracks all access and inserts the appropriate synchronisation barriers. Exemption 2: When implicit and explicit userspace coexist, the explicit side may skip intermediate synchronisation, and only place synchronisation barriers at transition points. For example, a Vulkan compositor displaying a buffer from an OpenGL application would need to synchronise its first access against the fence placed in the exclusive implicit-synchronisation slot. Once this read has fully retired, the compositor has no need to participate in implicit synchronisation until it is ready to return the buffer to the application, at which point it must insert all its non-retired accesses into the shared slot, which the application will then synchronise future write
Re: [Mesa-dev] [PATCH 0/6] dma-buf: Add an API for exporting sync files (v12)
Sorry for the mobile reply, but V4L2 is absolutely not write-only; there has never been an intersection of V4L2 supporting dmabuf and not supporting reads. I see your point about the heritage of dma_resv but it’s a red herring. It doesn’t matter who’s right, or who was first, or where the code was extracted from. It’s well defined that amdgpu defines resv to be one thing, that every other non-TTM user defines it to be something very different, and that the other TTM users define it to be something in the middle. We’ll never get to anything workable if we keep arguing who’s right. Everyone is wrong, because dma_resv doesn’t globally mean anything. It seems clear that there are three classes of synchronisation barrier (not using the ‘f’ word here), in descending exclusion order: - memory management barriers (amdgpu exclusive fence / ttm_bo->moving) - implicit synchronisation write barriers (everyone else’s exclusive fences, amdgpu’s shared fences) - implicit synchronisation read barriers (everyone else’s shared fences, also amdgpu’s shared fences sometimes) I don’t see a world in which these three uses can be reduced to two slots. What also isn’t clear to me though, is how the memory-management barriers can exclude all other access in the original proposal with purely userspace CS. Retaining the three separate modes also seems like a hard requirement to not completely break userspace, but then I don’t see how three separate slots would work if they need to be temporally ordered. amdgpu fixed this by redefining the meaning of the two slots, others fixed this by not doing one of the three modes. So how do we square the circle without encoding a DAG into the kernel? Do the two slots need to become a single list which is ordered by time + ‘weight’ and flattened whenever modified? Something else? Have a great weekend. -d > On 18 Jun 2021, at 5:43 pm, Christian König wrote: > > Am 18.06.21 um 17:17 schrieb Daniel Vetter: >> [SNIP] >> Ignoring _all_ fences is officially ok for pinned dma-buf. This is >> what v4l does. Aside from it's definitely not just i915 that does this >> even on the drm side, we have a few more drivers nowadays. > > No it seriously isn't. If drivers are doing this they are more than broken. > > See the comment in dma-resv.h > > * Based on bo.c which bears the following copyright notice, > * but is dual licensed: > > > > The handling in ttm_bo.c is and always was that the exclusive fence is used > for buffer moves. > > As I said multiple times now the *MAIN* purpose of the dma_resv object is > memory management and *NOT* synchronization. > > Those restrictions come from the original design of TTM where the dma_resv > object originated from. > > The resulting consequences are that: > > a) If you access the buffer without waiting for the exclusive fence you run > into a potential information leak. > We kind of let that slip for V4L since they only access the buffers for > writes, so you can't do any harm there. > > b) If you overwrite the exclusive fence with a new one without waiting for > the old one to signal you open up the possibility for userspace to access > freed up memory. > This is a complete show stopper since it means that taking over the > system is just a typing exercise. > > > What you have done by allowing this in is ripping open a major security hole > for any DMA-buf import in i915 from all TTM based driver. > > This needs to be fixed ASAP, either by waiting in i915 and all other drivers > doing this for the exclusive fence while importing a DMA-buf or by marking > i915 and all other drivers as broken. > > Sorry, but if you allowed that in you seriously have no idea what you are > talking about here and where all of this originated from. > > Regards, > Christian. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Linux Graphics Next: Userspace submission update
Hi Christian, On Tue, 1 Jun 2021 at 13:51, Christian König wrote: > Am 01.06.21 um 14:30 schrieb Daniel Vetter: > > If you want to enable this use-case with driver magic and without the > > compositor being aware of what's going on, the solution is EGLStreams. > > Not sure we want to go there, but it's definitely a lot more feasible > > than trying to stuff eglstreams semantics into dma-buf implicit > > fencing support in a desperate attempt to not change compositors. > > Well not changing compositors is certainly not something I would try > with this use case. > > Not changing compositors is more like ok we have Ubuntu 20.04 and need > to support that we the newest hardware generation. Serious question, have you talked to Canonical? I mean there's a hell of a lot of effort being expended here, but it seems to all be predicated on the assumption that Ubuntu's LTS HWE/backport policy is totally immutable, and that we might need to make the kernel do backflips to work around that. But ... is it? Has anyone actually asked them how they feel about this? I mean, my answer to the first email is 'no, absolutely not' from the technical perspective (the initial proposal totally breaks current and future userspace), from a design perspective (it breaks a lot of usecases which aren't single-vendor GPU+display+codec, or aren't just a simple desktop), and from a sustainability perspective (cutting Android adrift again isn't acceptable collateral damage to make it easier to backport things to last year's Ubuntu release). But then again, I don't even know what I'm NAKing here ... ? The original email just lists a proposal to break a ton of things, with proposed replacements which aren't technically viable, and it's not clear why? Can we please have some more details and some reasoning behind them? I don't mind that userspace (compositor, protocols, clients like Mesa as well as codec APIs) need to do a lot of work to support the new model. I do really care though that the hard-binary-switch model works fine enough for AMD but totally breaks heterogeneous systems and makes it impossible for userspace to do the right thing. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Linux Graphics Next: Userspace submission update
Hi, On Tue, 1 Jun 2021 at 14:18, Michel Dänzer wrote: > On 2021-06-01 2:10 p.m., Christian König wrote: > > Am 01.06.21 um 12:49 schrieb Michel Dänzer: > >> There isn't a choice for Wayland compositors in general, since there can > >> be arbitrary other state which needs to be applied atomically together > >> with the new buffer. (Though in theory, a compositor might get fancy and > >> special-case surface commits which can be handled by waiting on the GPU) Yeah, this is pretty crucial. > >> Latency is largely a matter of scheduling in the compositor. The latency > >> incurred by the compositor shouldn't have to be more than single-digit > >> milliseconds. (I've seen total latency from when the client starts > >> processing a (static) frame to when it starts being scanned out as low as > >> ~6 ms with https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1620, > >> lower than typical with Xorg) > > > > Well let me describe it like this: > > > > We have an use cases for 144 Hz guaranteed refresh rate. That essentially > > means that the client application needs to be able to spit out one > > frame/window content every ~6.9ms. That's tough, but doable. > > > > When you now add 6ms latency in the compositor that means the client > > application has only .9ms left for it's frame which is basically impossible > > to do. > > You misunderstood me. 6 ms is the lowest possible end-to-end latency from > client to scanout, but the client can start as early as it wants/needs to. > It's a trade-off between latency and the risk of missing a scanout cycle. Not quite. When weston-presentation-shm is reporting is a 6ms delta between when it started its rendering loop and when the frame was presented to screen. How w-p-s was run matters a lot, because you can insert an arbitrary delay in there to simulate client rendering. It also matters a lot that the client is SHM, because that will result in Mutter doing glTexImage2D on whatever size the window is, then doing a full GL compositing pass, so even if it's run with zero delay, 6ms isn't 'the amount of time it takes Mutter to get a frame to screen', it's measuring the overhead of a texture upload and full-screen composition as well. I'm assuming the 'guaranteed 144Hz' target is a fullscreen GL client, for which you definitely avoid TexImage2D, and could hopefully (assuming the client picks a modifier which can be displayed) also avoid the composition pass in favour of direct scanout from the client buffer; that would give you a much lower number. Each compositor has its own heuristics around timing. They all make their own tradeoff between low latency and fewest dropped frames. Obviously, the higher your latency, the lower the chance of missing a deadline. There's a lot of detail in the MR that Michel linked. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi, On Fri, 30 Apr 2021 at 10:35, Daniel Vetter wrote: > On Fri, Apr 30, 2021 at 11:08 AM Christian König > wrote: > > This doesn't work in hardware. We at least need to setup a few registers > > and memory locations from inside the VM which userspace shouldn't have > > access to when we want the end of batch fence and ring buffer start to > > be reliable. > > The thing is, we don't care whether it's reliable or not. Userspace is > allowed to lie, not signal, signal the wrong thing, out of order, > everything. > > The design assumes all this is possible. > > So unless you can't signal at all from userspace, this works. And for > the "can't signal at all" it just means something needs to do a cpu > busy wait and burn down lots of cpu time. I hope that's not your hw > design :-) I've been sitting this one out so far because what other-Dan's proposed seems totally sensible and workable for me, so I'll let him argue it rather than confuse it. But - yes. Our threat model does not care about a malicious content which deliberately submits garbage and then gets the compositor to display garbage. If that's the attack then you could just emit noise from your frag shader. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi, On Tue, 20 Apr 2021 at 20:30, Daniel Vetter wrote: > The thing is, you can't do this in drm/scheduler. At least not without > splitting up the dma_fence in the kernel into separate memory fences > and sync fences I'm starting to think this thread needs its own glossary ... I propose we use 'residency fence' for execution fences which enact memory-residency operations, e.g. faulting in a page ultimately depending on GPU work retiring. And 'value fence' for the pure-userspace model suggested by timeline semaphores, i.e. fences being (*addr == val) rather than being able to look at ctx seqno. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi, On Tue, 20 Apr 2021 at 20:03, Bas Nieuwenhuizen wrote: > On Tue, Apr 20, 2021 at 8:16 PM Daniel Stone wrote: > >> It's a jarring transition. If you take a very narrow view and say 'it's >> all GPU work in shared buffers so it should all work the same', then >> client<->winsys looks the same as client<->client gbuffer. But this is a >> trap. >> > > I think this is where I think we have have a serious gap of what a winsys > or a compositor is. Like if you have only a single wayland server running > on a physical machine this is easy. But add a VR compositor, an > intermediate compositor (say gamescope), Xwayland and some containers/VM, > some video capture (or, gasp, a browser that doubles as compositor) and > this story gets seriously complicated. Like who are you protecting from > who? at what point is something client<->winsys vs. client<->client? > As I've said upthread, the line is _seriously_ blurred, and is only getting less clear. Right now, DRI3 cannot even accept a dma_fence, let alone a drm_syncobj, let alone a memory fence. Crossing those boundaries is hard, and requires as much thinking as typing. That's a good thing. Conflating every synchronisation desire into a single userspace-visible primitive makes this harder, because it treats game threads the same as other game threads the same as VR compositors the same as embedding browsers the same as compositors etc. Drawing very clear lines between game threads and the external world, with explicit weakening as necessary, makes those jarring transitions of privilege and expectation clear and explicit. Which is a good thing, since we're trying to move away from magic and implicit. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi, On Tue, 20 Apr 2021 at 19:54, Daniel Vetter wrote: > So I can mostly get behind this, except it's _not_ going to be > dma_fence. That thing has horrendous internal ordering constraints > within the kernel, and the one thing that doesn't allow you is to make > a dma_fence depend upon a userspace fence. > > But what we can do is use the same currently existing container > objects like drm_syncobj or sync_file (timeline syncobj would fit best > tbh), and stuff a userspace fence behind it. The only trouble is that > currently timeline syncobj implement vulkan's spec, which means if you > build a wait-before-signal deadlock, you'll wait forever. Well until > the user ragequits and kills your process. > > So for winsys we'd need to be able to specify the wait timeout > somewhere for waiting for that dma_fence to materialize (plus the > submit thread, but userspace needs that anyway to support timeline > syncobj) if you're importing an untrusted timeline syncobj. And I > think that's roughly it. > Right. The only way you get to materialise a dma_fence from an execbuf is that you take a hard timeout, with a penalty for not meeting that timeout. When I say dma_fence I mean dma_fence, because there is no extant winsys support for drm_symcobj, so this is greenfield: the winsys gets to specify its terms of engagement, and again, we've been the orange/green-site enemies of users for quite some time already, so we're happy to continue doing so. If the actual underlying primitive is not a dma_fence, and compositors/protocol/clients need to eat a bunch of typing to deal with a different primitive which offers the same guarantees, then that's fine, as long as there is some tangible whole-of-system benefit. How that timeout is actually realised is an implementation detail. Whether it's a property of the last GPU job itself that the CPU-side driver can observe, or that the kernel driver guarantees that there is a GPU job launched in parallel which monitors the memory-fence status and reports back through a mailbox/doorbell, or the CPU-side driver enqueues kqueue work for $n milliseconds' time to check the value in memory and kill the context if it doesn't meet expectations - whatever. I don't believe any of those choices meaningfully impact on kernel driver complexity relative to the initial proposal, but they do allow us to continue to provide the guarantees we do today when buffers cross security boundaries. There might well be an argument for significantly weakening those security boundaries and shifting the complexity from the DRM scheduler into userspace compositors. So far though, I have yet to see that argument made coherently. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
On Tue, 20 Apr 2021 at 19:00, Christian König < ckoenig.leichtzumer...@gmail.com> wrote: > Am 20.04.21 um 19:44 schrieb Daniel Stone: > > But winsys is something _completely_ different. Yes, you're using the GPU > to do things with buffers A, B, and C to produce buffer Z. Yes, you're > using vkQueuePresentKHR to schedule that work. Yes, Mutter's composition > job might depend on a Chromium composition job which depends on GTA's > render job which depends on GTA's compute job which might take a year to > complete. Mutter's composition job needs to complete in 'reasonable' > (again, FSVO) time, no matter what. The two are compatible. > > How? Don't lump them together. Isolate them aggressively, and > _predictably_ in a way that you can reason about. > > What clients do in their own process space is their own business. Games > can deadlock themselves if they get wait-before-signal wrong. Compute jobs > can run for a year. Their problem. Winsys is not that, because you're > crossing every isolation boundary possible. Process, user, container, VM - > every kind of privilege boundary. Thus far, dma_fence has protected us from > the most egregious abuses by guaranteeing bounded-time completion; it also > acts as a sequencing primitive, but from the perspective of a winsys person > that's of secondary importance, which is probably one of the bigger > disconnects between winsys people and GPU driver people. > > > Finally somebody who understands me :) > > Well the question is then how do we get winsys and your own process space > together then? > It's a jarring transition. If you take a very narrow view and say 'it's all GPU work in shared buffers so it should all work the same', then client<->winsys looks the same as client<->client gbuffer. But this is a trap. Just because you can mmap() a file on an NFS server in New Zealand doesn't mean that you should have the same expectations of memory access to that file as you do to of a pointer from alloca(). Even if the primitives look the same, you are crossing significant boundaries, and those do not come without a compromise and a penalty. > Anyway, one of the great things about winsys (there are some! trust me) is > we don't need to be as hopelessly general as for game engines, nor as > hyperoptimised. We place strict demands on our clients, and we literally > kill them every single time they get something wrong in a way that's > visible to us. Our demands on the GPU are so embarrassingly simple that you > can run every modern desktop environment on GPUs which don't have unified > shaders. And on certain platforms who don't share tiling formats between > texture/render-target/scanout ... and it all still runs fast enough that > people don't complain. > > > Ignoring everything below since that is the display pipeline I'm not > really interested in. My concern is how to get the buffer from the client > to the server without allowing the client to get the server into trouble? > > My thinking is still to use timeouts to acquire texture locks. E.g. when > the compositor needs to access texture it grabs a lock and if that lock > isn't available in less than 20ms whoever is holding it is killed hard and > the lock given to the compositor. > > It's perfectly fine if a process has a hung queue, but if it tries to send > buffers which should be filled by that queue to the compositor it just gets > a corrupted window content. > Kill the client hard. If the compositor has speculatively queued sampling against rendering which never completed, let it access garbage. You'll have one frame of garbage (outdated content, all black, random pattern; the failure mode is equally imperfect, because there is no perfect answer), then the compositor will notice the client has disappeared and remove all its resources. It's not possible to completely prevent this situation if the compositor wants to speculatively pipeline work, only ameliorate it. From a system-global point of view, just expose the situation and let it bubble up. Watch the number of fences which failed to retire in time, and destroy the context if there are enough of them (maybe 1, maybe 100). Watch the number of contexts the file description get forcibly destroyed, and destroy the file description if there are enough of them. Watch the number of descriptions which get forcibly destroyed, and destroy the process if there are enough of them. Watch the number of processes in a cgroup/pidns which get forcibly destroyed, and destroy the ... etc. Whether it's the DRM driver or an external monitor such as systemd/Flatpak/podman/Docker doing that is pretty immaterial, as long as the concept of failure bubbling up remains. (20ms is objectively the wrong answer FWIW, because we're not a hard RTOS. But if our biggest point of disagreeme
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
On Tue, 20 Apr 2021 at 17:25, Marek Olšák wrote: > Daniel, imagine hardware that can only do what Windows does: future fences > signalled by userspace whenever userspace wants, and no kernel queues like > we have today. > > The only reason why current AMD GPUs work is because they have a ring > buffer per queue with pointers to userspace command buffers followed by > fences. What will we do if that ring buffer is removed? > I can totally imagine that; memory fences are clearly a reality and we need to make them work for functionality as well as performance. Let's imagine that winsys joins that flying-car future of totally arbitrary sync, that we work only on memory fences and nothing else, and that this all happens by the time we're all vaccinated and can go cram into a room with 8000 other people at FOSDEM instead of trying to do this over email. But the first couple of sentences of your proposal has the kernel monitoring those synchronisation points to ensure that they complete in bounded time. That already _completely_ destroys the purity of the simple picture you paint. Either there are no guarantees and userspace has to figure it out, or there are guarantees and we have to compromise that purity. I understand how you arrived at your proposal from your perspective as an extremely skilled driver developer who has delivered gigantic performance improvements to real-world clients. As a winsys person with a very different perspective, I disagree with you on where you are drawing the boundaries, to the point that I think your initial proposal is worse than useless; doing glFinish() or the VkFence equivalent in clients would be better in most cases than the first mail. I don't want to do glFinish (which I'm right about), and you don't want to do dma_fence (which you're right about). So let's work together to find a middle ground which we're both happy with. That middle ground does exist, and we as winsys people are happy to eat a significant amount of pain to arrive at that middle ground. Your current proposal is at once too gentle on the winsys, and far too harsh on it. I only want to move where and how those lines are drawn, not to pretend that all the world is still a single-context FIFO execution engine. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi, On Tue, 20 Apr 2021 at 16:46, Jason Ekstrand wrote: > It's still early in the morning here and I'm not awake yet so sorry if > this comes out in bits and pieces... > No problem, it's helpful. If I weren't on this thread I'd be attempting to put together a 73-piece chest of drawers whose instructions are about as clear as this so far, so I'm in the right head space anyway. > IMO, there are two problems being solved here which are related in > very subtle and tricky ways. They're also, admittedly, driver > problems, not really winsys problems. Unfortunately, they may have > winsys implications. > Yeah ... bingo. > First, is better/real timelines for Vulkan and compute. [...] > > We also want something like this for compute workloads. [...] > Totally understand and agree with all of this. Memory fences seem like a good and useful primitive here. > The second biting issue is that, in the current kernel implementation > of dma-fence and dma_resv, we've lumped internal synchronization for > memory management together with execution synchronization for > userspace dependency tracking. And we have no way to tell the > difference between the two internally. Even if user space is passing > around sync_files and trying to do explicit sync, once you get inside > the kernel, they're all dma-fences and it can't tell the difference. > Funny, because 'lumped [the two] together' is exactly the crux of my issues ... > If we move Stop here, because ... > to a more userspace-controlled synchronization model with > wait-before-signal and no timeouts unless requested, regardless of the > implementation, it plays really badly dma-fence. And, by "badly" I > mean the two are nearly incompatible. I would go further than that, and say completely, fundamentally, conceptually, incompatible. > From a user space PoV, it means > it's tricky to provide the finite time dma-fence guarantee. From a > kernel PoV, it's way worse. Currently, the way dma-fence is > constructed, it's impossible to deadlock assuming everyone follows the > rules. The moment we allow user space to deadlock itself and allow > those deadlocks to leak into the kernel, we have a problem. Even if > we throw in some timeouts, we still have a scenario where user space > has one linearizable dependency graph for execution synchronization > and the kernel has a different linearizable dependency graph for > memory management and, when you smash them together, you may have > cycles in your graph. > > So how do we sort this all out? Good question. It's a hard problem. > Probably the hardest problem here is the second one: the intermixing > of synchronization types. Solving that one is likely going to require > some user space re-plumbing because all the user space APIs we have > for explicit sync are built on dma-fence. > Gotcha. Firstly, let's stop, as you say, lumping things together. Timeline semaphores and compute's GPU-side spinlocks etc, are one thing. I accept those now have a hard requirement on something like memory fences, where any responsibility is totally abrogated. So let's run with that in our strawman: Vulkan compute & graphics & transfer queues all degenerate to something spinning (hopefully GPU-assisted gentle spin) on a uint64 somewhere. The kernel has (in the general case) no visibility or responsibility into these things. Fine - that's one side of the story. But winsys is something _completely_ different. Yes, you're using the GPU to do things with buffers A, B, and C to produce buffer Z. Yes, you're using vkQueuePresentKHR to schedule that work. Yes, Mutter's composition job might depend on a Chromium composition job which depends on GTA's render job which depends on GTA's compute job which might take a year to complete. Mutter's composition job needs to complete in 'reasonable' (again, FSVO) time, no matter what. The two are compatible. How? Don't lump them together. Isolate them aggressively, and _predictably_ in a way that you can reason about. What clients do in their own process space is their own business. Games can deadlock themselves if they get wait-before-signal wrong. Compute jobs can run for a year. Their problem. Winsys is not that, because you're crossing every isolation boundary possible. Process, user, container, VM - every kind of privilege boundary. Thus far, dma_fence has protected us from the most egregious abuses by guaranteeing bounded-time completion; it also acts as a sequencing primitive, but from the perspective of a winsys person that's of secondary importance, which is probably one of the bigger disconnects between winsys people and GPU driver people. Anyway, one of the great things about winsys (there are some! trust me) is we don't need to be as hopelessly general as for game engines, nor as hyperoptimised. We place strict demands on our clients, and we literally kill them every single time they get something wrong in a way that's visible to us. Our demands on the GPU are so
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi, On Tue, 20 Apr 2021 at 16:16, Christian König < ckoenig.leichtzumer...@gmail.com> wrote: > Am 20.04.21 um 17:07 schrieb Daniel Stone: > > If the compositor no longer has a guarantee that the buffer will be ready > for composition in a reasonable amount of time (which dma_fence gives us, > and this proposal does not appear to give us), then the compositor isn't > trying to use the buffer for compositing, it's waiting asynchronously on a > notification that the fence has signaled before it attempts to use the > buffer. > > Marek's initial suggestion is that the kernel signal the fence, which > would unblock composition (and presumably show garbage on screen, or at > best jump back to old content). > > My position is that the compositor will know the process has crashed > anyway - because its socket has been closed - at which point we destroy all > the client's resources including its windows and buffers regardless. > Signaling the fence doesn't give us any value here, _unless_ the compositor > is just blindly waiting for the fence to signal ... which it can't do > because there's no guarantee the fence will ever signal. > > > Yeah, but that assumes that the compositor has change to not blindly wait > for the client to finish rendering and as Daniel explained that is rather > unrealistic. > > What we need is a fallback mechanism which signals the fence after a > timeout and gives a penalty to the one causing the timeout. > > That gives us the same functionality we have today with the in software > scheduler inside the kernel. > OK, if that's the case then I think I'm really missing something which isn't explained in this thread, because I don't understand what the additional complexity and API change gains us (see my first reply in this thread). By way of example - say I have a blind-but-explicit compositor that takes a drm_syncobj along with a dmabuf with each client presentation request, but doesn't check syncobj completion, it just imports that into a VkSemaphore + VkImage and schedules work for the next frame. Currently, that generates an execbuf ioctl for the composition (ignore KMS for now) with a sync point to wait on, and the kernel+GPU scheduling guarantees that the composition work will not begin until the client rendering work has retired. We have a further guarantee that this work will complete in reasonable time, for some value of 'reasonable'. My understanding of this current proposal is that: * userspace creates a 'present fence' with this new ioctl * the fence becomes signaled when a value is written to a location in memory, which is visible through both CPU and GPU mappings of that page * this 'present fence' is imported as a VkSemaphore (?) and the userspace Vulkan driver will somehow wait on this value either before submitting work or as a possibly-hardware-assisted GPU-side wait (?) * the kernel's scheduler is thus eliminated from the equation, and every execbuf is submitted directly to hardware, because either userspace knows that the fence has already been signaled, or it will issue a GPU-side wait (?) * but the kernel is still required to monitor completion of every fence itself, so it can forcibly complete, or penalise the client (?) Lastly, let's say we stop ignoring KMS: what happens for the render-with-GPU-display-on-KMS case? Do we need to do the equivalent of glFinish() in userspace and only submit the KMS atomic request when the GPU work has fully retired? Clarifying those points would be really helpful so this is less of a strawman. I have some further opinions, but I'm going to wait until I understand what I'm actually arguing against before I go too far. :) The last point is very salient though. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
On Tue, 20 Apr 2021 at 15:58, Christian König < ckoenig.leichtzumer...@gmail.com> wrote: > Am 20.04.21 um 16:53 schrieb Daniel Stone: > > On Mon, 19 Apr 2021 at 11:48, Marek Olšák wrote: > >> Deadlock mitigation to recover from segfaults: >> - The kernel knows which process is obliged to signal which fence. This >> information is part of the Present request and supplied by userspace. >> - If the producer crashes, the kernel signals the submit fence, so that >> the consumer can make forward progress. >> - If the consumer crashes, the kernel signals the return fence, so that >> the producer can reclaim the buffer. >> - A GPU hang signals all fences. Other deadlocks will be handled like GPU >> hangs. >> > > Another thought: with completely arbitrary userspace fencing, none of this > is helpful either. If the compositor can't guarantee that a hostile client > has submitted a fence which will never be signaled, then it won't be > waiting on it, so it already needs infrastructure to handle something like > this. > > > That already handles the crashed-client case, because if the client > crashes, then its connection will be dropped, which will trigger the > compositor to destroy all its resources anyway, including any pending waits. > > > Exactly that's the problem. A compositor isn't immediately informed that > the client crashed, instead it is still referencing the buffer and trying > to use it for compositing. > If the compositor no longer has a guarantee that the buffer will be ready for composition in a reasonable amount of time (which dma_fence gives us, and this proposal does not appear to give us), then the compositor isn't trying to use the buffer for compositing, it's waiting asynchronously on a notification that the fence has signaled before it attempts to use the buffer. Marek's initial suggestion is that the kernel signal the fence, which would unblock composition (and presumably show garbage on screen, or at best jump back to old content). My position is that the compositor will know the process has crashed anyway - because its socket has been closed - at which point we destroy all the client's resources including its windows and buffers regardless. Signaling the fence doesn't give us any value here, _unless_ the compositor is just blindly waiting for the fence to signal ... which it can't do because there's no guarantee the fence will ever signal. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi, On Mon, 19 Apr 2021 at 11:48, Marek Olšák wrote: > Deadlock mitigation to recover from segfaults: > - The kernel knows which process is obliged to signal which fence. This > information is part of the Present request and supplied by userspace. > - If the producer crashes, the kernel signals the submit fence, so that > the consumer can make forward progress. > - If the consumer crashes, the kernel signals the return fence, so that > the producer can reclaim the buffer. > - A GPU hang signals all fences. Other deadlocks will be handled like GPU > hangs. > Another thought: with completely arbitrary userspace fencing, none of this is helpful either. If the compositor can't guarantee that a hostile client has submitted a fence which will never be signaled, then it won't be waiting on it, so it already needs infrastructure to handle something like this. That already handles the crashed-client case, because if the client crashes, then its connection will be dropped, which will trigger the compositor to destroy all its resources anyway, including any pending waits. GPU hangs also look pretty similar; it's an infinite wait, until the client resubmits a new buffer which would replace (& discard) the old. So signal-fence-on-process-exit isn't helpful and doesn't provide any extra reliability; it in fact probably just complicates things. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi, On Tue, 20 Apr 2021 at 13:01, Daniel Vetter wrote: > - We live in a post xf86-video-$vendor world, and all these other > compositors rely on implicit sync. You're not going to be able to get > rid of them anytime soon. What's worse, all the various EGL/vk buffer > sharing things also rely on implicit sync, so you get to fix up tons of > applications on top. Any plan that's realistic needs to cope with > implicit/explicit at the same time together won't work. > > - Absolute infuriating, but you can't use page-faulting together with any > dma_fence synchronization primitives, whether implicit or explicit. This > means until the entire ecosystem moved forward (good luck with that) we > have to support dma_fence. The only sync model that works together with > page faults is userspace fence based sync. > This should get rid of the oversync issues, and since implicit sync is > backed in everywhere right now, you'll have to deal with implicit sync for > a very long time. > Depends what you mean by 'implicit sync'. ;) Getting userspace (Vulkan WSI, EGL, Wayland compositors, browsers, media clients) over to explicit sync is easy, _provided_ that the explicit sync gives us the same guarantees as implicit sync, i.e. completes in bounded time, GPU/display work can be flushed to the kernel predicated on fence completion with the kernel handling synchronisation and scheduling. It's just a matter of typing, and until now we haven't had a great reason to do that typing. Now we do have that reason, so we are implementing it. Whether it's dma_fence or drm_syncobj is mostly immaterial; we can encode in protocol requirements that you can't try to use wait-before-signal with drm_syncobj and you'll get killed if you try. Getting that userspace over to fully userspace-based sync (wait-before-signal or wait-never-signal, no kernel assistance but you just have to roll your own polling or signal handling on either CPU or GPU side) is not easy. It might never happen, because it's an extraordinary amount of work, introduces a huge amount of fragility into a super-critical path, and and so far it's not clear that it's a global performance improvement for the whole system, just shifting performance problems from kernel to userspace, and probably (AFAICT) making them worse in addition to the other problems it brings. What am I missing? Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal
Hi Marek, On Mon, 19 Apr 2021 at 11:48, Marek Olšák wrote: > *2. Explicit synchronization for window systems and modesetting* > > The producer is an application and the consumer is a compositor or a > modesetting driver. > > *2.1. The Present request* > So the 'present request' is an ioctl, right? Not a userspace construct like it is today? If so, how do we correlate the two? The terminology is pretty X11-centric so I'll assume that's what you've designed against, but Wayland and even X11 carry much more auxiliary information attached to a present request than just 'this buffer, this swapchain'. Wayland latches a lot of data on presentation, including non-graphics data such as surface geometry (so we can have resizes which don't suck), window state (e.g. fullscreen or not, also so we can have resizes which don't suck), and these requests can also cascade through a tree of subsurfaces (so we can have embeds which don't suck). X11 mostly just carries timestamps, which is more tractable. Given we don't want to move the entirety of Wayland into kernel-visible objects, how do we synchronise the two streams so they aren't incoherent? Taking a rough stab at it whilst assuming we do have DRM_IOCTL_NONMODE_PRESENT, this would create a present object somewhere in kernel space, which the producer would create and ?? export a FD from, that the compositor would ?? import. As part of the Present request, the producer will pass 2 fences (sync > objects) to the consumer alongside the presented DMABUF BO: > - The submit fence: Initially unsignalled, it will be signalled when the > producer has finished drawing into the presented buffer. > We have already have this in Wayland through dma_fence. I'm relaxed about this becoming drm_syncobj or drm_newmappedysncobjthing, it's just a matter of typing. X11 has patches to DRI3 to support dma_fence, but they never got merged because it was far too invasive to a server which is no longer maintained. > - The return fence: Initially unsignalled, it will be signalled when the > consumer has finished using the presented buffer. > Currently in Wayland the return fence (again a dma_fence) is generated by the compositor and sent as an event when it's done, because we can't have speculative/empty/future fences. drm_syncobj would make this possible, but so far I've been hesitant because I don't see the benefit to it (more below). > Deadlock mitigation to recover from segfaults: > - The kernel knows which process is obliged to signal which fence. This > information is part of the Present request and supplied by userspace. > Same as today with dma_fence. Less true with drm_syncobj if we're using timelines. > - If the producer crashes, the kernel signals the submit fence, so that > the consumer can make forward progress. > This is only a change if the producer is now allowed to submit a fence before it's flushed the work which would eventually fulfill that fence. Using dma_fence has so far isolated us from this. > - If the consumer crashes, the kernel signals the return fence, so that > the producer can reclaim the buffer. > 'The consumer' is problematic, per below. I think the wording you want is 'if no references are held to the submitted present object'. > - A GPU hang signals all fences. Other deadlocks will be handled like GPU > hangs. > > Other window system requests can follow the same idea. > Which other window system requests did you have in mind? Again, moving the entirety of Wayland's signaling into the kernel is a total non-starter. Partly because it means our entire protocol would be subject to the kernel's ABI rules, partly because the rules and interdependencies between the requests are extremely complex, but mostly because the kernel is just a useless proxy: it would be forced to do significant work to reason about what those requests do and when they should happen, but wouldn't be able to make those decisions itself so would have to just punt everything to userspace. Unless we have eBPF compositors. > Merged fences where one fence object contains multiple fences will be > supported. A merged fence is signalled only when its fences are signalled. > The consumer will have the option to redefine the unsignalled return fence > to a merged fence. > An elaboration of how this differed from drm_syncobj would be really helpful here. I can make some guesses based on the rest of the mail, but I'm not sure how accurate they are. > *2.2. Modesetting* > > Since a modesetting driver can also be the consumer, the present ioctl > will contain a submit fence and a return fence too. One small problem with > this is that userspace can hang the modesetting driver, but in theory, any > later present ioctl can override the previous one, so the unsignalled > presentation is never used. > This is also problematic. It's not just KMS, but media codecs too - V4L doesn't yet have explicit fencing, but given the programming model of codecs and how deeply they interoperate, but
Re: [Mesa-dev] piglit default main branch results - Re: Rename "master" branch to "main"?
On Tue, 30 Mar 2021 at 01:34, Ilia Mirkin wrote: > https://cgit.freedesktop.org/piglit > > This still displays the "master" branch by default. I think you're > supposed to change ... something ... in the repo, which indicates > which is the default branch. > *jedi handwave* no it doesn't > On Mon, Mar 29, 2021 at 8:31 PM Jordan Justen > wrote: > > I have only seen one issue so far for the piglit default branch > > change. It appears that the piglit patchwork scripting isn't > > configured for the main branch. I think it's long past time to retire > > the piglit patchwork, so I made a request to the admins to do that. I > > think Mesa already retired patchwork. *jedi handwave* it was always retired, nothing to see here, move along please *foom* ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] docs: consistent language
On Tue, 16 Mar 2021 at 13:08, Jason Ekstrand wrote: > On March 16, 2021 05:06:53 Daniel Stone wrote: > >> On Mon, 15 Mar 2021 at 20:54, Jason Ekstrand >> wrote: >> >>> Three comments: >>> >>> 1. +1 >>> 2. Khronos has generally standardized on American English in their >>> specs so I guess that provides some sort of prior art or something. >>> 3. I'd generally be a fan of encouraging American spellings in common >>> code as well. As someone who doesn't use auto-complete, having to >>> remember which variable is named "color" and which is "colour" would >>> get painful. Fortunately, we don't have a whole lot of variation here >>> but it'd be good to keep consistency. >> >> >> 4. UK English is unarguably correct and better. >> > > > > 5. That doesn't change point #3 in particular; just like coding style, one >> 'bad' choice is better than no choice at all. >> > > I have three words for you: three space tabs. Yeah... It may be a good > style but consistency is good. 浪 > Absolutely. Apparently I forgot to write the final line which is that I think standardising on US English across the board is a good thing, even though I personally like it about as much as three-space tabs. As you can see from the previous sentence. ;) So, +1 from me too. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] docs: consistent language
On Mon, 15 Mar 2021 at 20:54, Jason Ekstrand wrote: > Three comments: > > 1. +1 > 2. Khronos has generally standardized on American English in their > specs so I guess that provides some sort of prior art or something. > 3. I'd generally be a fan of encouraging American spellings in common > code as well. As someone who doesn't use auto-complete, having to > remember which variable is named "color" and which is "colour" would > get painful. Fortunately, we don't have a whole lot of variation here > but it'd be good to keep consistency. 4. UK English is unarguably correct and better. 5. That doesn't change point #3 in particular; just like coding style, one 'bad' choice is better than no choice at all. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Rust drivers in Mesa
Hi, On Fri, 2 Oct 2020 at 08:31, Kristian Kristensen wrote: > On Fri, Oct 2, 2020 at 8:14 AM Dave Airlie wrote: >> My feeling is the pieces that would benefit the most are the things >> touch the real world, GLSL compiler, SPIR-V handling, maybe some of >> the GL API space, but I also feel these are the messiest things to >> move to rust. I'm not sure you'd get much benefit from vulkan API >> space or touching kernel interfaces as much. Granted a backend >> compiler might be the easiest place to experiment. >> >> I think the problem is you really want to justify that whatever you >> introduce using rust is sufficiently better along any axis than c/c++ >> to justify the enhanced build time requirements and maintenance, >> otherwise someone will just rewrite it in C/C++ to avoid having to >> pull a rust compiler into their distro requires. > > > Just as a data point, in ChromeOS we already build core parts of the OS with > rust. Our hypervisor (crosvm) is written in rust and using rust is generally > encouraged. That's just one distro, of course, but my sense is that rust > isn't going away - parts of GNOME are moving to rust as well - and it's a > good fit for a project like mesa. If we could agree that a rust dependency in > a gallium backend isn't any worse than a C++ backend (not talking about > subjective qualities of the languages, just build time deps), then it's not > so much a matter of "justifying" the use of rust, but a developer preference. > When we write a new compiler backend in modern C++, I don't think that > there's unanimous agreement that that's better than C along all axises, so > it's a bit of a high bar to raise for rust. > > I would also worry less about which components supposedly would theoretically > benefit the most from being written in rust. The GLSL front might be better > in rust, but rewriting it is a big investment and will introduce more bugs > and regressions before it gets to that hypothetical ideal state, for a > codebase that's otherwise in stasis. I think it makes more sense to apply the > benefits of rust to live code - code that we work on and write every day, > since that way we get to actually benefit from the features of the language. > I'm in favor of figuring out how to integrate rust code in mesa and maybe > rewrite a small helper library or binary to rust or incrementally start > converting a compiler backend or driver. Yep. Before we can land a single bit of code, we need to do a bunch of annoying things like build-system integration, FFI bridging, agreeing on conventions and style, etc etc. Trying to do that whilst also replacing the GLSL compiler or vtn is way too much; it's 100% doomed to failure, even if they're the highest-value targets on some axes. If you start with a leaf node somewhere greenfield (like a new Vulkan driver, or a new experimental backend), it makes it much easier to approach, because you've cut down on the number of moving parts (e.g. maintaining APIs in two directions), you don't have to replace something which already exists and people don't want to regress, and it also allows people who don't really care about it to ignore it until you've got all the impedance mismatches and teething pain figured out. After that you'll have a reasonable template for how you bridge the two language worlds and how you handle particular patterns, which others can replicate and improve on, than having to bridge the entire codebase in one go just to get anything working. But even then, there's a lot more to Rust than 'woo now everything is secure'. Complex state trackers like the GL API implementation could get a bunch of help from the type system; things like backends can get a lot smaller as collect/iterate/match/transform is much better supported by the language than it is in C. So the benefit in that case, apart from developer ergonomy of not having to write the same trivial patterns over and over again, is that the important parts of code becomes a lot easier to reason about. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Rename "master" branch to "main"?
Hi, On Mon, 3 Aug 2020 at 17:16, Jason Ekstrand wrote: > On Mon, Aug 3, 2020 at 11:12 AM Kenneth Graunke wrote: > > Seems reasonable to me...in the old Subversion days, we called it > > 'trunk'...then 'master' with Git...but calling the main development > > branch 'main' is arguably the simplest and most descriptive term. > > > > One thing we'll have to coordinate: getting Gitlab CI / Marge and the > > Intel Mesa CI to switch over at the right time, so we don't end up > > breaking/interrupting those services. Should be easy, just requires > > a bit of coordination. > > Yup, I threw Daniel onto the CC of this e-mail explicitly for that > reason. We may also want to coordinate with the rest of fd.o so that > everyone chooses the same new mainline branch name. I just added > Michel to the CC as he's doing lots of CI stuff and might be a good > person to help coordinate there. I certainly don't want to pull the > rug out from under anyone. That's fine by me. I think 'main' is a perfectly fine name, and we'd be happy to encode that in whatever useful way. I suppose the main problem with a global nature is the very disparate nature of the projects - getting everyone to move at once before we throw the switch would require a great deal of effort. But we can figure it out. As for retargeting MRs; if it can be done manually, then it can be done automatically as well. We can figure out a way to just automatically retarget all the outstanding MRs, but a couple of weeks' leadtime would be good right now. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] New Mesa3D.org website proposal
Hi, On Fri, 29 May 2020 at 10:08, Erik Faye-Lund wrote: > In the light of the explanation above, do you still have objections to > this split? > > Obviously, we haven't moved forward yet ;-) Well, we have now after getting some agreement. Please enjoy your shiny new https://www.mesa3d.org and https://docs.mesa3d.org experience. Brian, could you please change the DNS for mesa3d.org and www.mesa3d.org to point to 35.227.58.183? No need to change archive.mesa3d.org, that should still point to the old site. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] New Mesa3D.org website proposal
Hi, I'm generally ambivalent on which day it moves, though depending on how late in the day it comes out, it might not actually be Thursday - if the release comes out late at night, I'm more likely to finish up the migration over the weekend. On Wed, 13 May 2020 at 13:43, Brian Paul wrote: > On 05/13/2020 03:13 AM, Erik Faye-Lund wrote: > > 2. Some people have been asking me why the website is set up as a > > separate repo rather than a subdirectory inside the main mesa repo. > > > > The quick answer to that is that it seemed more logical that way to me. > > The website isn't really tied to the mesa release-schedule, apart from > > linking to new releases. It's really its own thing. We might for > > instance want to post updates when other projects in the mesa-group > > cuts a release. So this seems to give us some more freedom. > > > > If someone has good arguments against this, I'm open to changing it. > > But this seems like the best option to me. > > I'd really like to keep the Mesa content as part of the main Mesa repo. > I didn't realize you did that. The website is part of the project and > it's more convenient to have it all in one place. It's a fair point, but then there's the ageing issue with branches. If someone is working against the 19.3.x repo, they'll see a snapshot of the website at a pretty arbitrary point in time - whenever it was that it was forked. That doesn't sound ideal to me, nor does the alternate idea of picking back all the content to all our live branches (e.g. when we do a 20.0.x stable release, cherry-pick the news-page update to 19.3.x?). There's also a circular dependency issue related to publishing releases: it seems odd to commit and tag a release, then have the very next commit be the update to the page, which you can't do in one as you don't know the SHA before you commit. Code information and documentation _definitely_ belongs with the Mesa project though. Things like the Gallium3D documentation published out to ReadTheDocs should be maintained in with the code and generated with the code - and this should be extended through the whole Mesa source tree, so we have much better live docs of our codebase. I've been working with Erik to figure out how we can best get this published alongside the main website. We absolutely shouldn't be putting up any barriers to keeping the website updated and making sure we have good contributions to it, though. I think putting it in a separate repo actually makes it slightly easier if anything, provided that we have a good set of reviewers keeping eyes on it, and clear instructions on how to contribute to it. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] New Mesa3D.org website proposal
Hi Brian, On Fri, 8 May 2020 at 15:30, Brian Paul wrote: > Done. easydns says it may take up to 3 hours to go into effect. Thanks for doing this! Could you please add the following TXT records as well (note that they're FQDN, so you might want to chop the trailing '.mesa3d.org' from the first segment: _gitlab-pages-verification-code.mesa3d.org TXT gitlab-pages-verification-code=e8a33eb47c034b08afd339cb3a801d88 _gitlab-pages-verification-code.www.mesa3d.org TXT gitlab-pages-verification-code=28d457c44c75d4a1e460243ded2b4311 _gitlab-pages-verification-code.docs.mesa3d.org TXT gitlab-pages-verification-code=6045fd12252cc01f5891fa3f81b140fe archive.mesa3d.org is now live, but I'll be in touch with you to cut over the main website once we've done some more infrastructure transitional things: moving websites with live TLS is somewhat difficult ... Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] New Mesa3D.org website proposal
Hi, On Thu, 7 May 2020 at 18:08, Erik Faye-Lund wrote: > On Thu, 2020-05-07 at 11:03 -0600, Brian Paul wrote: > > It seems like only the front page is served at the moment. Is it > > possible to get a look at the rest? The front page looks nice. > > Yeah, we need to set up docs.mesa3d.org first. I haven't bothered > setting up the temporary URLs yet, but my current preview is hosted > here: After the nginx infrastructure work over the past few weeks, we can now actually set up redirects. So I think the better thing would be to assume this will be www.mesa3d.org hosted on Pages, then have redirects over for tarball downloads. Brian, could you please add a DNS entry for archive.mesa3d.org as a CNAME to annarchy.freedesktop.org, and docs.mesa3d.org as a CNAME to gitlab.freedesktop.org? If you can do that, I'll get the archive hosting set up separately, then when we're ready we can cut over www.mesa3d.org and mesa3d.org to Pages. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Drop scons for 20.1?
Hi, On Tue, 25 Feb 2020 at 23:42, Kristian Høgsberg wrote: > It's been a while since Dylan did the work to make meson support > Windows and there's been plenty of time to provide feedback or improve > argue why we still need scons. I haven't seen any such discussion and > I think we've waited long enough. Dylan's work didn't get enabled by default, and also depended on Windows runners having a system environment set up. Taking a little bit of work from the GStreamer side, I wrote a native Windows VS2019 CI which also uses Docker to build in: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4304 Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
Hi, On Mon, 16 Mar 2020 at 15:33, Tomek Bury wrote: > > GL and GLES are not relevant. What is relevant is EGL, which defines > > interfaces to make things work on the native platform. > Yes and no. This is what EGL spec says about sharing a texture between > contexts: Contexts are different though ... > There are similar statements with regards to the lack of > synchronisation guarantees for EGL images or between GL and native > rendering, etc. This also isn't about native rendering. > But the main thing here is that EGL and Vulkan differ > significantly. Sure, I totally agree. > The eglSwapBuffers() is expected to post an unspecified > "back buffer" to the display system using some internal driver magic. > EGL driver is then expected to obtain another back buffer at some > unspecified point in the future. Yes, this is rather the point: EGL doesn't specify platform-related 'black magic' to make things just work, because that's part of the platform implementation details. And, as things stand, on Linux one of those things is implicit synchronisation, unless the desired end state of your driver is no synchronisation. This thread is a discussion about changing that. > > If you are using EGL_WL_bind_wayland_display, then one of the things > > it is explicitly allowed/expected to do is to create a Wayland > > protocol interface between client and compositor, which can be used to > > pass buffer handles and metadata in a platform-specific way. Adding > > synchronisation is also possible. > Only one-way synchronisation is possible with this mechanism. There's > a standard protocol for recycling buffers - wl_buffer_release() so > buffer hand-over from the compositor to client remains unsynchronised > - see below. That's not true; you can post back a sync token every time the client buffer is used by the compositor. > > > The most troublesome part was Wayland buffer release mechanism, as it > > > only involves a CPU signalling over Wayland IPC, without any 3D driver > > > involvement. The choices were: explicit synchronisation extension or a > > > buffer copy in the compositor (i.e. compositor textures from the copy, so > > > the client can re-write the original), or some implicit synchronisation > > > in kernel space (but that wasn't an option in Broadcom driver). > > > > You can add your own explicit synchronisation extension. > I could but that requires implementing in in the driver and in a > number of compositors, therefore a standard extension > zwp_linux_explicit_synchronization_v1 is much better choice here than > a custom one. EGL_WL_bind_wayland_display is explicitly designed to allow each driver to implement its own private extensions without modifying compositors. For instance, Mesa adds the `wl_drm` extension, which is used for bidirectional communication between the EGL implementations in the client and compositor address spaces, without modifying either. > > In every cross-process and cross-subsystem usecase, synchronisation is > > obviously required. The two options for this are to implement kernel > > support for implicit synchronisation (as everyone else has done), > That would require major changes in driver architecture or a 2nd > mechanisms doing the same thing but in kernel space - both are > non-starters. OK. As it stands, everyone else has the kernel mechanism (e.g. via dmabuf resv), so in this case if you are reinventing the underlying platform in a proprietary stack, you get to solve the same problems yourselves. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
Hi Tomek, On Mon, 16 Mar 2020 at 12:55, Tomek Bury wrote: > I've been wrestling with the sync problems in Wayland some time ago, but only > with regards to 3D drivers. > > The guarantee given by the GL/GLES spec is limited to a single graphics > context. If the same buffer is accessed by 2 contexts the outcome is > unspecified. The cross-context and cross-process synchronisation is not > guaranteed. It happens to work on Mesa, because the read/write locking is > implemented in the kernel space, but it didn't work on Broadcom driver, which > has read-write interlocks in user space. GL and GLES are not relevant. What is relevant is EGL, which defines interfaces to make things work on the native platform. EGL doesn't define any kind of synchronisation model for the Wayland, X11, or GBM/KMS platforms - but it's one of the things which has to work. It doesn't say that the implementation must make sure that the requested format is displayable, but you sort of take it for granted that if you ask EGL to display something it will do so. Synchronisation is one of those mechanisms which is left to the platform to implement under the hood. In the absence of platform support for explicit synchronisation, the synchronisation must be implicit. > A Vulkan client makes it even worse because of conflicting requirements: > Vulkan's vkQueuePresentKHR() passes in a number of semaphores but disallows > waiting. Wayland WSI requires wl_surface_commit() to be called from > vkQueuePresentKHR() which does require a wait, unless a synchronisation > primitive representing Vulkan samaphores is passed between Vulkan client and > the compositor. If you are using EGL_WL_bind_wayland_display, then one of the things it is explicitly allowed/expected to do is to create a Wayland protocol interface between client and compositor, which can be used to pass buffer handles and metadata in a platform-specific way. Adding synchronisation is also possible. > The most troublesome part was Wayland buffer release mechanism, as it only > involves a CPU signalling over Wayland IPC, without any 3D driver > involvement. The choices were: explicit synchronisation extension or a buffer > copy in the compositor (i.e. compositor textures from the copy, so the client > can re-write the original), or some implicit synchronisation in kernel space > (but that wasn't an option in Broadcom driver). You can add your own explicit synchronisation extension. In every cross-process and cross-subsystem usecase, synchronisation is obviously required. The two options for this are to implement kernel support for implicit synchronisation (as everyone else has done), or implement generic support for explicit synchronisation (as we have been working on with implementations inside Weston and Exosphere at least), or implement private support for explicit synchronisation, or do nothing and then be surprised at the lack of synchronisation. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] gitlab.fd.o financial situation and impact on services
Hi Jan, On Fri, 28 Feb 2020 at 10:09, Jan Engelhardt wrote: > On Friday 2020-02-28 08:59, Daniel Stone wrote: > >I believe that in January, we had $2082 of network cost (almost > >entirely egress; ingress is basically free) and $1750 of > >cloud-storage cost (almost all of which was download). That's based > >on 16TB of cloud-storage (CI artifacts, container images, file > >uploads, Git LFS) egress and 17.9TB of other egress (the web service > >itself, repo activity). Projecting that out [×12 for a year] gives > >us roughly $45k of network activity alone, > > I had come to a similar conclusion a few years back: It is not very > economic to run ephemereal buildroots (and anything like it) between > two (or more) "significant locations" of which one end is located in > a Large Cloud datacenter like EC2/AWS/etc. > > As for such usecases, me and my surrounding peers have used (other) > offerings where there is 50 TB free network/month, and yes that may > have entailed doing more adminning than elsewhere - but an admin > appreciates $2000 a lot more than a corporation, too. Yes, absolutely. For context, our storage & network costs have increased >10x in the past 12 months (~$320 Jan 2019), >3x in the past 6 months (~$1350 July 2019), and ~2x in the past 3 months (~$2000 Oct 2019). I do now (personally) think that it's crossed the point at which it would be worthwhile paying an admin to solve the problems that cloud services currently solve for us - which wasn't true before. Such an admin could also deal with things like our SMTP delivery failure rate, which in the past year has spiked over 50% (see previous email), demand for new services such as Discourse which will enable user support without either a) users having to subscribe to a mailing list, or b) bug trackers being cluttered up with user requests and other non-bugs, etc. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Intel-gfx] gitlab.fd.o financial situation and impact on services
On Fri, 28 Feb 2020 at 10:06, Erik Faye-Lund wrote: > On Fri, 2020-02-28 at 11:40 +0200, Lionel Landwerlin wrote: > > Yeah, changes on vulkan drivers or backend compilers should be > > fairly > > sandboxed. > > > > We also have tools that only work for intel stuff, that should never > > trigger anything on other people's HW. > > > > Could something be worked out using the tags? > > I think so! We have the pre-defined environment variable > CI_MERGE_REQUEST_LABELS, and we can do variable conditions: > > https://docs.gitlab.com/ee/ci/yaml/#onlyvariablesexceptvariables > > That sounds like a pretty neat middle-ground to me. I just hope that > new pipelines are triggered if new labels are added, because not > everyone is allowed to set labels, and sometimes people forget... There's also this which is somewhat more robust: https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2569 Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Intel-gfx] gitlab.fd.o financial situation and impact on services
On Fri, 28 Feb 2020 at 08:48, Dave Airlie wrote: > On Fri, 28 Feb 2020 at 18:18, Daniel Stone wrote: > > The last I looked, Google GCP / Amazon AWS / Azure were all pretty > > comparable in terms of what you get and what you pay for them. > > Obviously providers like Packet and Digital Ocean who offer bare-metal > > services are cheaper, but then you need to find someone who is going > > to properly administer the various machines, install decent > > monitoring, make sure that more storage is provisioned when we need > > more storage (which is basically all the time), make sure that the > > hardware is maintained in decent shape (pretty sure one of the fd.o > > machines has had a drive in imminent-failure state for the last few > > months), etc. > > > > Given the size of our service, that's a much better plan (IMO) than > > relying on someone who a) isn't an admin by trade, b) has a million > > other things to do, and c) hasn't wanted to do it for the past several > > years. But as long as that's the resources we have, then we're paying > > the cloud tradeoff, where we pay more money in exchange for fewer > > problems. > > Admin for gitlab and CI is a full time role anyways. The system is > definitely not self sustaining without time being put in by you and > anholt still. If we have $75k to burn on credits, and it was diverted > to just pay an admin to admin the real hw + gitlab/CI would that not > be a better use of the money? I didn't know if we can afford $75k for > an admin, but suddenly we can afford it for gitlab credits? s/gitlab credits/GCP credits/ I took a quick look at HPE, which we previously used for bare metal, and it looks like we'd be spending $25-50k (depending on how much storage you want to provision, how much room you want to leave to provision more storage later, how much you care about backups) to run a similar level of service so that'd put a bit of a dint in your year-one budget. The bare-metal hosting providers also add up to more expensive than you might think, again especially if you want either redundancy or just backups. > > Yes, we could federate everything back out so everyone runs their own > > builds and executes those. Tinderbox did something really similar to > > that IIRC; not sure if Buildbot does as well. Probably rules out > > pre-merge testing, mind. > > Why? does gitlab not support the model? having builds done in parallel > on runners closer to the test runners seems like it should be a thing. > I guess artifact transfer would cost less then as a result. It does support the model but if every single build executor is also compiling Mesa from scratch locally, how long do you think that's going to take? > > Again, if you want everything to be centrally > > designed/approved/monitored/controlled, that's a fine enough idea, and > > I'd be happy to support whoever it was who was doing that for all of > > fd.o. > > I don't think we have any choice but to have someone centrally > controlling it, You can't have a system in place that lets CI users > burn largs sums of money without authorisation, and that is what we > have now. OK, not sure who it is who's going to be approving every update to every .gitlab-ci.yml in the repository, or maybe we just have zero shared runners and anyone who wants to do builds can BYO. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [Intel-gfx] gitlab.fd.o financial situation and impact on services
On Fri, 28 Feb 2020 at 03:38, Dave Airlie wrote: > b) we probably need to take a large step back here. > > Look at this from a sponsor POV, why would I give X.org/fd.o > sponsorship money that they are just giving straight to google to pay > for hosting credits? Google are profiting in some minor way from these > hosting credits being bought by us, and I assume we aren't getting any > sort of discounts here. Having google sponsor the credits costs google > substantially less than having any other company give us money to do > it. The last I looked, Google GCP / Amazon AWS / Azure were all pretty comparable in terms of what you get and what you pay for them. Obviously providers like Packet and Digital Ocean who offer bare-metal services are cheaper, but then you need to find someone who is going to properly administer the various machines, install decent monitoring, make sure that more storage is provisioned when we need more storage (which is basically all the time), make sure that the hardware is maintained in decent shape (pretty sure one of the fd.o machines has had a drive in imminent-failure state for the last few months), etc. Given the size of our service, that's a much better plan (IMO) than relying on someone who a) isn't an admin by trade, b) has a million other things to do, and c) hasn't wanted to do it for the past several years. But as long as that's the resources we have, then we're paying the cloud tradeoff, where we pay more money in exchange for fewer problems. > If our current CI architecture is going to burn this amount of money a > year and we hadn't worked this out in advance of deploying it then I > suggest the system should be taken offline until we work out what a > sustainable system would look like within the budget we have, whether > that be never transferring containers and build artifacts from the > google network, just having local runner/build combos etc. Yes, we could federate everything back out so everyone runs their own builds and executes those. Tinderbox did something really similar to that IIRC; not sure if Buildbot does as well. Probably rules out pre-merge testing, mind. The reason we hadn't worked everything out in advance of deploying is because Mesa has had 3993 MRs in the not long over a year since moving, and a similar number in GStreamer, just taking the two biggest users. At the start it was 'maybe let's use MRs if you want to but make sure everything still goes through the list', and now it's something different. Similarly the CI architecture hasn't been 'designed', so much as that people want to run dEQP and Piglit on their hardware pre-merge in an open fashion that's actually accessible to people, and have just done it. Again, if you want everything to be centrally designed/approved/monitored/controlled, that's a fine enough idea, and I'd be happy to support whoever it was who was doing that for all of fd.o. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] gitlab.fd.o financial situation and impact on services
Hi Matt, On Thu, 27 Feb 2020 at 23:45, Matt Turner wrote: > We're paying 75K USD for the bandwidth to transfer data from the > GitLab cloud instance. i.e., for viewing the https site, for > cloning/updating git repos, and for downloading CI artifacts/images to > the testing machines (AFAIU). I believe that in January, we had $2082 of network cost (almost entirely egress; ingress is basically free) and $1750 of cloud-storage cost (almost all of which was download). That's based on 16TB of cloud-storage (CI artifacts, container images, file uploads, Git LFS) egress and 17.9TB of other egress (the web service itself, repo activity). Projecting that out gives us roughly $45k of network activity alone, so it looks like this figure is based on a projected increase of ~50%. The actual compute capacity is closer to $1150/month. > I was not aware that we were being charged for anything wrt GitLab > hosting yet (and neither was anyone on my team at Intel that I've > asked). This... kind of needs to be communicated. > > A consistent concern put forth when we were discussing switching to > GitLab and building CI was... how do we pay for it. It felt like that > concern was always handwaved away. I heard many times that if we > needed more runners that we could just ask Google to spin up a few > more. If we needed testing machines they'd be donated. No one > mentioned that all the while we were paying for bandwidth... Perhaps > people building the CI would make different decisions about its > structure if they knew it was going to wipe out the bank account. The original answer is that GitLab themselves offered to sponsor enough credit on Google Cloud to get us started. They used GCP themselves so they could assist us (me) in getting bootstrapped, which was invaluable. After that, Google's open-source program office offered to sponsor us for $30k/year, which was I believe last April. Since then the service usage has increased roughly by a factor of 10, so our 12-month sponsorship is no longer enough to cover 12 months. > What percentage of the bandwidth is consumed by transferring CI > images, etc? Wouldn't 75K USD would be enough to buy all the testing > machines we need and host them within Google or wherever so we don't > need to pay for huge amounts of bandwidth? Unless the Google Cloud Platform starts offering DragonBoards, it wouldn't reduce our bandwidth usage as the corporate network is treated separately for egress. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Unable to find a matching EGL config on Wayland
Hi Devashish, It sounds like your application, as well as eglinfo, are not even trying to use Wayland. Maybe they are autodetecting the platform incorrectly and trying to use GBM instead. This could perhaps be solved by setting the $WAYLAND_DISPLAY environment variable to the name of the socket Weston uses (usually 'wayland-0'). However you should probably contact The Qt Company for support in using Qt, and you should contact TI (or your BSP vendor) for support with the proprietary Imagination EGL/GLES driver. Cheers, Daniel On Fri, 31 Jan 2020 at 08:12, Devashish Tiwari wrote: > > Hello, > > > I have built a simple application using QT & SDL that has a button which > opens a preview on clicking, whenever camera (system/usb) is connected to the > EVM. > > My app successfully receives correct parameters related to camera > (resolution, frame rate, frame format) and I have verified it using debug > prints. However, when I click on preview button, it crashes and throws error > related to EGL. > > > On executing it with gdb, I found out that it is not able to get a valid EGL > configuration. The eglChooseConfig function (in SDL_egl.c) doesn't find a > matching EGL config and it fails to create a surface and open the preview. > > Also, I do not get any "EGL 1.4 Initialised" prints when my app executes. > This print, on the contrary, is visible when I run OGLES2ChameleonMan in > /usr/bin/SGX/demos/Wayland. > > > On digging further, I found that the "eglinfo" command displays the egl > configurations on the terminal. > > However, when it returns EGL_BAD_ALLOC on my EVM, if I run it without > stopping weston. It runs successfully only after stopping weston. I am > assuming this is the reason why my application is not able to find a matching > EGL config because it runs over weston. > > --- > > root@am57xx-evm:/mnt# eglinfo > > eglGetConfigs(): > == > ** EGL config number: 0 EGL vendor string: > Imagination Technologies EGL version string: 1.4 build 1.14@3699939 (MAIN) > EGL extensions: EGL_IMG_client_api_ogl EGL_KHR_image EGL_KHR_image_base > EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image > EGL_KHR_gl_renderbuffer_image EGL_KHR_vg_parent_image EGL_IMG_cl_image > EGL_KHR_fence_sync EGL_IMG_context_priority EGL_IMG_hibernate_process > EGL_IMG_image_plane_attribs EGL_KHR_surfaceless_context EGL_KHR_wait_sync > EGL_KHR_create_context EGL_WL_bind_wayland_display > EGL_EXT_image_dma_buf_import EGL client APIs are: OpenGL_ES EGL config > Attributes: EGL_CONFIG_ID = 0x1 EGL_BUFFER_SIZE = 0x20 EGL_RED_SIZE = 0x8 > EGL_GREEN_SIZE = 0x8 EGL_BLUE_SIZE = 0x8 EGL_LUMINANCE_SIZE = 0x0 > EGL_ALPHA_SIZE = 0x8 EGL_ALPHA_MASK_SIZE = 0x0 EGL_BIND_TO_TEXTURE_RGB = 0x1 > EGL_BIND_TO_TEXTURE_RGBA = 0x1 EGL_COLOR_BUFFER_TYPE = 0x308e EGL_CONFI G_CAVEAT = 0x3038 EGL_CONFORMANT = 0x1 EGL_DEPTH_SIZE = 0x18 EGL_LEVEL = 0x0 EGL_MAX_PBUFFER_WIDTH = 0x1000 EGL_MAX_PBUFFER_HEIGHT = 0x1000 EGL_MAX_PBUFFER_PIXELS = 0x100 EGL_MAX_SWAP_INTERVAL = 0xa EGL_MIN_SWAP_INTERVAL = 0x1 EGL_NATIVE_RENDERABLE = 0x0 EGL_NATIVE_VISUAL_ID = 0x0 EGL_NATIVE_VISUAL_TYPE = 0x0 EGL_RENDERABLE_TYPE = 0x1 EGL_OPENGL_ES_BIT EGL_SAMPLE_BUFFERS = 0x0 EGL_SAMPLES = 0x0 EGL_STENCIL_SIZE = 0x8 EGL_SURFACE_TYPE = 0x5 EGL_PBUFFER_BIT EGL_WINDOW_BIT EGL_TRANSPARENT_TYPE = 0x3038 EGL_TRANSPARENT_RED_VALUE = 0x0 EGL_TRANSPARENT_GREEN_VALUE = 0x0 EGL_TRANSPARENT_BLUE_VALUE = 0x0 Creating Window surface.. PVR:(Error): WSEGL_CreateWindowDrawable: Couldn't set CRTC: Permission denied [0, ] Unable to create surface egl error 'EGL_BAD_ALLOC' (0x3003) > --- > > > Does anyone has any idea about this strange behaviour? > > Also, Is there any way to run eglinfo command without stopping weston? > Because I think my application is polling available egl configurations at the > run time, but is not returning anything as it runs over weston. > > > The ldd of my application returns: > > --- > > root@am57xx-evm:/mnt# ldd cam_preview > linux-vdso.so.1 (0xbec5) > libpjsua.so.2 => /usr/local/PJSIP_ONLY/libpjsua.so.2 (0xb6e6e000) > libpjsip-ua.so.2 => /usr/local/PJSIP_ONLY/libpjsip-ua.so.2 (0xb6e48000) > libpjsip-simple.so.2 => /usr/local/PJSIP_ONLY/libpjsip-simple.so.2 > (0xb6e28000) > libpjsip.so.2 => /usr/local/PJSIP_ONLY/libpjsip.so.2 (0xb6dda000) > libpjmedia-codec.so.2 => /usr/local/PJSIP_ONLY/libpjmedia-codec.so.2 > (0xb6dba000) > libpjmedia-videodev.so.2 => /usr/local/PJSIP_ONLY/libpjmedia-videodev.so.2 > (0xb6d9e000) > libpjmedia-audiodev.so.2 => /usr/local/PJSIP_ONLY/libpjmedia-audiodev.so.2 > (0xb6d88000) > libpjmedia.so.2 => /usr/local/PJSIP_ONLY/libpjmedia.so.2 (0xb6d12000) >
Re: [Mesa-dev] marge currently down
Hi, On Fri, 3 Jan 2020 at 16:57, Eric Anholt wrote: > marge broke over the holidays due to a gitlab upgrade. It's unclear > when we'll get it back up -- daniels might do a downgrade but is > currently out sick > > https://gitlab.freedesktop.org/freedesktop/freedesktop/issues/228 I've pushed a fix to our GitLab instance, and Marge is now happy again. Please resume using it. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Requiring a full author name when contributing to mesa?
On Wed, 11 Dec 2019 at 22:35, Timothy Arceri wrote: > So it seems lately we have been increasingly merging patches with made > up names, or single names etc [1]. The latest submitted patch has the > name Icecream95. This seems wrong to me from a point of keeping up the > integrity of the project. I'm not a legal expert but it doesn't seem > ideal to be amassing commits with these type of author tags from that > point of view either. > > Is it just me or do others agree we should at least require a proper > name on the commits (as fake as that may be also)? Seems like a low bar > to me. What benefit does it bring? Icecream95 could just resubmit as 'John Johnson'; would we just take that as face value that that was their 'real name' and accept the contribution? I know someone in Australia who changed their name via deed poll to Stormy Wrathcauser (changed slightly to protect their privacy, but very close). Would we accept their contribution if they posted, or would we have to stop and take measures to verify that that was their real legal name? What about Chinese contributors, who as noted in thread tend to use made-up non-legal pseudonyms anyway? Unless we're actually trying to bring up and enforce a web of trust, I don't think there's any point in requiring that the submitter's name conforms to some notion of idealised naming - it's just window dressing. I also don't see any point in trying to enforce a web of trust. Debian's method of doing this involves a hundred people standing around in a room looking at drivers' licenses from countries they might not have even heard of before to verify identity. But I'm certainly not an expert at identifying whether or not a Bolivian drivers' license which is put in front of my face is forged or not, and suspect no-one on this list is. If someone is determined to compromise the legal integrity of Mesa's codebase, requiring that they register as Juan Molinos or any other name which seems like it could be 'legitimate' is not really any barrier to entry. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] Revert "st/dri2: Implement DRI2bufferDamageExtension"
On Tue, 1 Oct 2019 at 10:24, Boris Brezillon wrote: > This reverts commit 492ffbed63a2a62759224b1c7d45aa7923d8f542. > > BACK_LEFT attachment can be outdated when the user calls > KHR_partial_update(), leading to a damage region update on the > wrong pipe_resource object. > Let's not expose the ->set_damage_region() method until the core is > fixed to handle that properly. > > Cc: mesa-sta...@lists.freedesktop.org > Signed-off-by: Boris Brezillon Acked-by: Daniel Stone ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH RFC 2/2] dri: Pass a __DRIcontext to ->set_damage_region()
Hi Boris, On Tue, 1 Oct 2019 at 09:25, Boris Brezillon wrote: > On Mon, 2 Sep 2019 16:32:01 +0200 Michel Dänzer wrote: > > On 2019-08-30 7:00 p.m., Boris Brezillon wrote: > > > So, next question is, do you think it's acceptable to pass a > > > DRIcontext here, and if not, do you have any idea how to solve this > > > problem? > > > > Hmm, not sure. Maybe it would be better to explicitly pass in the > > __DRIimage* to which the damage region applies? > > Sorry, for the late reply. I had a look at this proposal and I don't see > how passing a __DRIimage object would help. There's this comment [1] > that makes me think passing a drawable is the right thing to do, but at > the same time I'm not sure how to rework the logic to make it work > without having access to the pipe_context (sounds like an invasive > change to me). > So, I suggest that we revert [2] and [3] until we find a proper > solution to address the problem. > > Daniel, Qiang, are you okay with that? Yeah, I'm fine with that. I can't remember the details of why passing a DRIimage directly doesn't help. Writing that down somewhere would be good, I think. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Moving libglvnd to FreeDesktop Gitlab
Hi Kyle, On Fri, 27 Sep 2019 at 17:06, Kyle Brenneman wrote: > Okay, I've imported the Github repository here: > https://gitlab.freedesktop.org/glvnd/libglvnd That's great, thanks! I've given Adam Jackson access as requested (go to the 'members' tab from the glvnd group page). It sounds like it would be good to give Matt access, as well. Another thing to consider would be a CONTRIBUTING.md file, similar to the ones in Wayland/Weston, setting out clear expectations for how people should contribute, review, and ultimately merge code. > It looks like the code, bug reports, and pull requests all came in cleanly. > The release tarballs didn't, though. > > I guess the next step is to figure out what to do with the Github repository. > I've seen Github repos that are explicitly labeled as being a mirror of > another repo, but I'll have to go figure out how to configure that. https://gitlab.freedesktop.org/glvnd/libglvnd/-/settings/repository You can set up a push mirror here using an SSH key or password. > There's also some still-active discussions on the Github bug reports that > we'll need to migrate over. What we've done everywhere else is to post a link to the matching GitLab issue, note that it should be further discussed there, and then closed. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Release workflow with gitlab issues
On Wed, 18 Sep 2019 at 20:43, Mark Janes wrote: > Adam Jackson writes: > > Strictly, the "Reporter" access level and above can manage labels, > > milestones require "Developer" or above. Not super meaningful since the > > mesa group really only has Developer or above. > > > > I'm not super worried about it to be honest, people tend not to be > > malicious. > > I agree that malicious users will be less of a problem with gitlab > issues. I am somewhat worried about inadvertent mistakes. "I thought I > was in the browser tab with my forked repo..." > > That particular mistake would be hard to make, but it's hard to > anticipate accidents. > > When you delete a label, gitlab warns that you can't get it back. > Labels will hold a lot of information for us. We keep daily backups. Obviously it would be an enormous pain to deal with, but we could get the label information back if we really needed to. I have a strong suspicion that the bugs keep their association to the label, so we could probably even pull it out with some local Ruby, without resorting to backups. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Moving libglvnd to FreeDesktop Gitlab
Hi, On Wed, 4 Sep 2019 at 15:12, Chuck Atkins wrote: > Can we use Gitlab's GitHub import feature? > > https://gitlab.freedesktop.org/help/user/project/import/github.md > > I haven't used it before but it looks like it will migrate everything, i.e. > repo, issues, prs, etc. Yeah, we definitely can. We can create a new namespace for GLVND and import the project into there. Who else should I add to the group? Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Moving libglvnd to FreeDesktop Gitlab
Hi, On Sat, 31 Aug 2019 at 20:34, Matt Turner wrote: > Getting patches into libglvnd has proven quite difficult (see [0] for > example). There was some talk of moving it to FreeDesktop Gitlab on > IRC recently. Can we move forward with that? Are there objections to > doing so? We'd be happy to have it in on fd.o. Moving the vendor-neutral dispatch into a vendor-neutral space is probably a good move, even without the difficulty in having things merged. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH v2 2/2] panfrost: protect access to shared bo cache and transient pool
Hi Boris, On Sat, 31 Aug 2019 at 18:33, Boris Brezillon wrote: > On Sat, 31 Aug 2019 17:06:30 +0100 Daniel Stone wrote: > > On Fri, 30 Aug 2019 at 17:00, Rohan Garg wrote: > > > Both the BO cache and the transient pool are shared across > > > context's. Protect access to these with mutexes. > > > > These fixes seem right to me, and (minus the issues Boris pointed > > out), both are: > > Reviewed-by: Daniel Stone > > Unfortunately, they've already been applied (maybe I should wait > a bit longer before applying patches :-/). No problem, they'd already been reviewed in a previous round anyway. :) Have a great weekend! Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH v2 3/3] panfrost: Stop passing a ctx to functions being passed a batch
Hi Boris, On Sat, 31 Aug 2019 at 08:53, Boris Brezillon wrote: > @@ -123,8 +123,7 @@ struct panfrost_batch * > panfrost_job_create_batch(struct panfrost_context *ctx); > > void > -panfrost_job_free_batch(struct panfrost_context *ctx, > -struct panfrost_batch *batch); > +panfrost_job_free_batch(struct panfrost_batch *batch); > > struct panfrost_batch * > panfrost_job_get_batch(struct panfrost_context *ctx, > @@ -149,18 +148,16 @@ panfrost_flush_jobs_reading_resource(struct > panfrost_context *panfrost, > struct pipe_resource *prsc); > > void > -panfrost_job_submit(struct panfrost_context *ctx, struct panfrost_batch > *batch); > +panfrost_job_submit(struct panfrost_batch *batch); > > void > -panfrost_job_set_requirements(struct panfrost_context *ctx, > - struct panfrost_batch *batch); > +panfrost_job_set_requirements(struct panfrost_batch *batch); > > mali_ptr > panfrost_job_get_polygon_list(struct panfrost_batch *batch, unsigned size); > > void > -panfrost_job_clear(struct panfrost_context *ctx, > - struct panfrost_batch *batch, > +panfrost_job_clear(struct panfrost_batch *batch, > unsigned buffers, > const union pipe_color_union *color, > double depth, unsigned stencil); The series looks good to me and is: Reviewed-by: Daniel Stone But as a follow-on, it seems like most of the panfrost_job_*() functions should probably be panfrost_batch_*() to reduce confusion. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] panfrost: Make transient allocation rely on the BO cache
Hi Boris,
On Sat, 31 Aug 2019 at 11:47, Boris Brezillon
wrote:
> Right now, the transient memory allocator implements its own BO caching
> mechanism, which is not really needed since we already have a generic
> BO cache. Let's simplify things a bit.
>
> [...]
>
> bool fits_in_current = (batch->transient_offset + sz) <
> TRANSIENT_SLAB_SIZE;
>
> +if (likely(batch->transient_bo && fits_in_current)) {
> +/* We can reuse the current BO, so get it */
> +[...]
> } else {
> -/* Create a new BO and reference it */
> -bo = panfrost_drm_create_bo(screen, ALIGN_POT(sz, 4096), 0);
> +size_t bo_sz = sz < TRANSIENT_SLAB_SIZE ?
> + TRANSIENT_SLAB_SIZE : ALIGN_POT(sz, 4096);
Should we record the size of the allocated transient BO here, so we
can use the whole BO for transients when we allocate more than the
slab size?
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH v2 2/2] panfrost: protect access to shared bo cache and transient pool
Hi Rohan, On Fri, 30 Aug 2019 at 17:00, Rohan Garg wrote: > Both the BO cache and the transient pool are shared across > context's. Protect access to these with mutexes. These fixes seem right to me, and (minus the issues Boris pointed out), both are: Reviewed-by: Daniel Stone I think it might be worthwhile pushing both of these to the context in the longer term. I wonder if we should also rename the 'BO cache' to something else. (If only so I can stop saying the wrong thing about 'the BO cache'! Maybe we could call it 'BO reuse cache' or similar?) Another use of BO caching (usually 'BO handle cache') is to handle the exclusivity requirement around GEM handles. For any given kernel buffer (wherever it comes from: a panfrost_gem_object allocated by us, a panfrost_gem_object allocated by another process, a V4L2 buffer, etc), for every DRM connection (a drm_file represents one open() of the DRM device), there can only be one GEM handle referring to that buffer. drmPrimeFDToHandle() is where you will most often bump into this. If a client imports the same underlying buffer twice, drmPrimeFDToHandle() may return a GEM handle which you already have open. Since creating and destroying GEM handles is not refcounted, this means that you may get two panfrost_bo's referring to the same GEM handle. When you close one of those, the other will be left hanging: either you cannot use it (because the handle is invalid), or you will end up overlapping with another buffer which later comes to use the same GEM handle. Neither result is good. Given that, on the screen (not on the context, since it needs to be scoped to every user of the same DRM FD), we need a BO handle cache. The screen would have a list of every panfrost_bo for that device, and when panfrost_drm_import_bo() receives a GEM handle back from the kernel, we need to walk the list of all BOs. If we find a panfrost_bo with the same handle, we just bump the refcount and return that. This would need a screen-wide lock on the BO handle list. Would you be able to take a look at that please? Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Switching to Gitlab Issues instead of Bugzilla?
Hi, On Thu, 29 Aug 2019 at 21:35, Chris Wilson wrote: > Quoting Kristian Høgsberg (2019-08-29 21:20:12) > > On Thu, Aug 29, 2019 at 12:44 PM Chris Wilson > > wrote: > > > Quoting Kenneth Graunke (2019-08-29 19:52:51) > > > > - Moving bug reports between the kernel and Mesa would be harder. > > > > We would have to open a bug in the other system. (Then again, > > > > moving bugs between Mesa and X or Wayland would be easier...) > > > > > > All that I ask is that we move the kernel bugzilla along with it. Trying > > > to keep abreast of the bugs in the whole stack is important. Fwiw, the > > > kernel contains the > > > https:bugs.freedesktop.org/enter_bug.cgi?product=DRI > > > URL so would need some redirection for a few years... > > > > Would Rob's suggestion of creating a placeholder drm kernel repo for > > the purpose of issue tracking work for you? We can definitely get placeholder DRM repos and move those issues too. And set up redirects. (To be honest, rather than one-by-one redirects, my thought with Bugzilla was just to statically scrape all the bug pages and leave them as read-only static copies frozen in amber for time eternal. I'd welcome help from anyone who had the time to figure out the scripting and Apache runes required to make those be proper HTTP redirects, but am definitely not going to be able to do that myself.) > I think so. I just want a list of all bugs that may affect the code I'm > working on, wherever they were filed. I have a search in bugs.fdo, I > just need instructions on how to get the same from gitlab, hopefully in > a compact format. It's not clear to me what you need. Can you please give more details? If you want cross-component search results in a single list, that's not really something we can do today, and I don't know if it would land any time soon. You can however subscribe to particular issue labels, and when you see something that catches your eye add a 'todo' for it, then the main UI shows all your outstanding todos, including where people have mentioned you etc. > The issue URL will also need to be stable so that we can include it in > commits. From a glance, > https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/861, > looks like it will be adjusted if it ever gets moved. Sorry, 'moved' how? If you mean moved between components, yes the issue will move and there will be a new URL for that. However, going to that old URL will auto-redirect you to the new one. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Mesa GitLab <-> AppVeyor integration
Hi, On Wed, 28 Aug 2019 at 11:18, Michel Dänzer wrote: > On 2019-08-28 3:08 a.m., Eric Engestrom wrote: > > On Tuesday, 2019-08-27 13:31:22 +, Jose Fonseca wrote: > >> Appveyor seems to be building other MR 1781: > >> > >> https://ci.appveyor.com/project/mesa3d/mesa-re1yd/builds/26989425 > >> https://ci.appveyor.com/project/mesa3d/mesa-re1yd/history > >> https://gitlab.freedesktop.org/eric/mesa/pipelines/59190 > > > > You shouldn't take my MRs as an example for this, as I've set up the > > hook on my account, so my MRs always get picked up by appveyor :) > > Yeah, the external integration settings are per GitLab project, and > pre-merge CI pipelines for MRs run in the source project context, so the > appveyor integration would need to be set up in each forked project used > for MRs. > > This is a bit unfortunate, as it means the CI pipeline which runs (in > the main project context) after an MR is merged could fail at the > appveyor step, even if the pre-merge pipeline passed. > > Not sure what can be done about this though, other than requiring forked > projects used for MRs to set up the appveyor integration as well. To be honest, given the relative immaturity of external-job reporting, and some of the problems reported in #freedesktop for NetworkManager using it, I would probably suggest not using it. For Panfrost CI, which executes on an external system (LAVA), we create a new job in the GitLab CI pipeline with the 'idle-jobs' tag, which submits an HTTP request to LAVA to schedule the job, then sits waiting for it to report results and logs back. The 'idle-jobs' runner can do something like 64 concurrent jobs, as it's supposed to only be used for things like this. That might help with some of the flakiness, and bring the result reporting in line with the rest of execution. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Difference between TransformFeedback Gallium <-> Vulkan (V3D)
Hi Andreas, On Sun, 25 Aug 2019 at 21:11, Andreas Bergmeier wrote: > For a few weeks now I am working on implementing Vulkan for VideoCore 6 AKA > 42 (using V3D/DRM). Don't hold you breath ;) Great! I can't say much about the specifics of VideoCore hardware, but at least for some of the common parts ... > Currently I am trying to understand what is necessary or how to interact with > V3D. So I am looking at TransformFeedback because it interacts with quite a > few other parts of the pipeline and still seems less mangled into the big > logic than other features. So I am comparing how Gallium (V3D) is handling TF > in the state tracker VS how Vulkan (Intel) is handling the Extension. > > The following is what I so far think I gathered: > 1. GV3D is handling TransformFeedback directly with other bound parts of the > pipeline (e.g. `emit_state` in _v3dx_emit.c_). It seems to look into the > shader and associated TF specs. It seems to use "streamout targets", although > I do not yet understand what these are really. Then it passes all the > offsets, indices and finally resources to V3D. 'Stream out' is basically just what DirectX calls its version of transform feedback. The idea is the same: capturing output from vertex/geometry stages. > 2. The Vulkan Extension only knows about CounterBuffers and iterates over > these. Intel seems to call TF -> XF? and subsequently the buffers XFB. Have > also not yet gathered what is the difference and what all the gazillion > acronyms mean. 'XFB' is the most widely-used acronym for transform, as 'trans' or 'cross' can abbreviate to X. 'TF' is not as widely used as XFB. > So far my idea would be to implement TF similar to Intel and instead of > iterating over "streamout targets" I would iterate XFBs. So these would really be the same thing. A streamout target is where the VC4 writes its output stream of data from these shading stages, and a counter buffer is where Vulkan writes the output stream of data from these shading stages. > The problem with this approach is, that it will not be easy to mimic > `cl_emit` calls similar to Gallium. > My question now is which parts of V3D emits have a dependency. > > I would assume that I can move TRANSFORM_FEEDBACK_SPECS and > TRANSFORM_FEEDBACK_ENABLE to cmd state flush in Vulkan. > `vkCmdBeginTransformFeedbackEXT` shoudl then only need > TRANSFORM_FEEDBACK_BUFFER and TRANSFORM_FEEDBACK_OUTPUT_ADDRESS. > > Sorry if this is a bit confusing - I am really just trying to figure this out > one by one. This is getting more into the specifics of how you've structured the driver, as well as hardware specifics, but it sounds about right to me. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [MR] Update README to recommend MRs instead of `git send-email`
Hi, On Sat, 6 Jul 2019 at 18:39, Ilia Mirkin wrote: > I see this as driving away contributions, esp from new people. MR's > are annoying to create, since they require dealing with the hosting > provider, getting it to host clones, etc. Everyone has email. My position - and the evidence of velocity from projects who have switched - is already pretty clear, but you might be interested to read that even kernel.org admins are trying to move away from email: https://people.kernel.org/monsieuricon/patches-carved-into-developer-sigchains Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH 3/3] panfrost: Add support for modifiers
Hi,
On Fri, 5 Jul 2019 at 14:51, Alyssa Rosenzweig
wrote:
> > + switch(whandle->modifier) {
> > + case DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_ROCKCHIP):
> > + rsc->layout = PAN_AFBC;
> > + break;
>
> Why ROCKCHIP in particular? AFBC fourcc codes are based on modifiers,
> and there are a *lot* of them. We need to figure out which upstream
> modifier PAN_AFBC actually enables and use that:
>
> [...]
>
> On other AFBC flags, I don't know. I'm also not sure if Arm would chime
> in one way or the other.
>
> Mainly the issue is that whandle->modifier could contain some other,
> non-Rockchip modifier, and then the whole stack collapses; AFBC
> modifiers are meant to be parsedd, not =='d. See the komeda driver in
> the kernel for reference.
Yes and no. See below.
> > + if (want_afbc || (is_renderable && can_afbc && !is_texture))
>
> This doesn't seem quite correct. We preselect AFBC if:
>
> - AFBC is explicitly requested, OR
> - We're rendering to something that's not bound as a texture
> that could be AFBC
>
> In case one, what happens if AFBC is explicitly requested but we don't
> support AFBC on that format? It should fallback to linear, or error out.
>
> In case two, what if the modifier for linear or tiled is explicitly set?
> Now we've given them AFBC when they've asked for something else;
> switching it up is only safe for internal BOs.
>
> The logic in this code block has only ever been for internal BOs, where
> it's up to *us* to pick a format. Imports are logically distinct, since
> it's now up to the modifiers. Two different problems, unless the
> modifier is a "don't care".
It's still up to you-the-driver to pick a modifier.
EGL/Wayland/KMS all advertise sets of modifiers to clients. In
general, clients don't try to reason about the contents of those lists
- if they have multiple lists, they might intersect them, or they
might create their own list-of-one to be LINEAR, or whatever.
Modifier lists are explicitly unsorted. The list provided is the
modifiers which are acceptable to use. The driver is expected to sort
(on whatever metric) for 'optimality'. If none of the modifiers
provided are usable, then resource creation is free to fail. If the
provided modifiers are not optimal, then the driver just has to select
that less-optimal modifier. The only no-no is to select a modifier
which is not in the provided list.
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH 2/3] panfrost: Allocate scanout BOs in panfrost device
On Fri, 5 Jul 2019 at 14:38, Alyssa Rosenzweig wrote: > > +bool should_tile = !is_streaming && is_texture && is_2d && > > !is_scanout; > > I'm not opposed, but why can't we tile PIPE_BIND_SHARED textures? lima > does exactly that. If we can't tile them, we certainly can't AFBC them. We can't tile USE_SHARED unless it's explicitly listed in the modifiers. If we don't have modifiers, then USE_SHARED means we'd be sending it to a different process or subsystem (KMS, V4L, whatever) which isn't going to know that it's tiled, because we aren't using modifiers to tell it. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH 3/3] panfrost: Add support for modifiers
Hi, On Thu, 4 Jul 2019 at 09:47, Tomeu Vizoso wrote: > On Thu, 4 Jul 2019 at 10:05, Tomeu Vizoso wrote: > > -pres->layout = should_tile ? PAN_TILED : PAN_LINEAR; > > + if (want_afbc || (is_renderable && can_afbc && !is_texture)) > > +pres->layout = PAN_AFBC; > > We regress here because we seem to have some bugs regarding AFBC usage > with some formats such as rgba4 and rgb5_a1: > > https://gitlab.freedesktop.org/tomeu/mesa/-/jobs/399237 > > I'm looking at only enabling AFBC when explicitly asked to by the winsys. Again, the winsys just pulls the list of acceptable format/modifier pairs from the EGL format query. If Panfrost declares 1555+AFBC to be a supported combination, then the winsys will pass that on for clients to use. If it isn't working, my suggestion is to just never advertise it in the first place. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH 3/3] panfrost: Add support for modifiers
Hi Tomeu, On Thu, 4 Jul 2019 at 09:05, Tomeu Vizoso wrote: > @@ -362,14 +392,19 @@ panfrost_resource_create_bo(struct panfrost_screen > *screen, struct panfrost_reso > > /* Tiling textures is almost always faster, unless we only use it > once */ > > +bool can_afbc = panfrost_format_supports_afbc(res->format); > +bool want_afbc = > drm_find_modifier(DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_ROCKCHIP), > modifiers, count); > +bool want_linear = drm_find_modifier(DRM_FORMAT_MOD_LINEAR, > modifiers, count); > > [...] > -bool should_tile = !is_streaming && is_texture && is_2d && > !is_scanout; > +bool should_tile = !is_streaming && is_texture && is_2d && > !want_linear && !is_scanout; I don't think this is right. The list of modifiers we pass in is the total list of acceptable modifiers. Clients don't make any judgement or discernment about them, but just pass in the full list. For example, in Weston we use EGL to query the list of all modifiers supported by the driver, and then give those to the client as the list of acceptable modifiers; the EGL Wayland winsys will then pass that list into DRIImage create. This means that 'want_linear' will be true as long as Panfrost (or, on GBM, the KMS driver) supports linear, which realistically is all the time. I think ultimately we want should_tile to be (in order of evaluation): * definitively false if we can't support it - GPU doesn't support it, isn't a 2D texture/RT, etc * if the above passes, definitively true if the above conditions are met and we have passed in a modifier set which includes AFBC * if we can do it (first test) but we have no modifiers, definitively false if it's a shared/scanout buffer (i.e. we need to export the buffer somewhere else but the client isn't modifier-aware), or definitively true if it's just an internal buffer That code seems pretty familiar from VC4 (which is in turn probably from freedreno? everything else is). It could be semi-useful to look at how VC4 has structured its resource creation decision tree for modifiers, but on the other hand they also have an implicit fallback mode; when modifiers aren't available, it will pass the tiling information through a magic kernel side-channel, which we really don't want to do here. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH kmscube] Search for a suitable config
Hi Drew, On Wed, 3 Jul 2019 at 08:16, Drew DeVault wrote: > Instead of assuming the first will be suitable. kmscube fails to start > for me without this change. There are a couple of unrelated changes combined in here, but I think the core one is good. eglChooseConfig has some really useful properties, where it strictly specifies a sort order, does not include EGL_NATIVE_VISUAL_ID in that sort order, and silently accepts attributes which are not relevant for the sort. Your change which takes all possible configs and then queries them to make sure that the native visual ID is actually correct is the right thing to do. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH v4 5/5] panfrost: Add support for KHR_partial_update()
Hi,
To be honest, I haven't been able to look too closely at this one. I
wasn't able to easily reason about the twists and turns, so had to run
away to reviews elsewhere. But as long as we reload every single
region passed in - be it individually or just lazily pulling in the
extents, it's correct.
There's no need to intersect the partial_update region with the
scissor, since rendering outside of the partial_update area is
explicitly undefined.
On Wed, 26 Jun 2019 at 18:51, Alyssa Rosenzweig
wrote:
> > +binfo.src.resource = binfo.dst.resource =
> > ctx->pipe_framebuffer.cbufs[0]->texture;
> > +binfo.src.level = binfo.dst.level = 0;
> > +binfo.src.box.x = binfo.dst.box.x = rect->x;
> > +binfo.src.box.y = binfo.dst.box.y = rect->y;
> > +binfo.src.box.width = binfo.dst.box.width = rect->width;
> > +binfo.src.box.height = binfo.dst.box.height = rect->height;
> >
> > /* This avoids an assert due to missing nir_texop_txb support */
> > //binfo.src.box.depth = binfo.dst.box.depth = 1;
>
> This will need to be rebased in a slightly messy way, since
> panfrost_blit_wallpaper was edited pretty heavily in the mipmap series
> that just landed. Sorry for the conflicts, although conceptually this
> looks good.
>
> Have you considered if this interacts with mipmapping, by the way? I
> suppose surfaces that get partial updates *by definition* are not
> mipmapped, so that's an easy "who cares?" :)
Yeah, partial_update only applies to winsys surfaces.
> > +u_box_2d(batch->minx, batch->miny, damage.minx - batch->minx,
> > + batch->maxy - batch->miny, [0]);
> > +u_box_2d(damage.maxx, batch->miny, batch->maxx - damage.maxx,
> > + batch->maxy - batch->miny, [1]);
> > +u_box_2d(damage.minx, batch->miny, damage.maxx - damage.minx,
> > + damage.miny - batch->miny, [2]);
> > +u_box_2d(damage.minx, damage.maxy, damage.maxx - damage.minx,
> > + batch->maxy - damage.maxy, [3]);
> > +
> > +for (unsigned i = 0; i < 4; i++) {
> > +if (!rects[i].width || !rects[i].height)
> > +continue;
>
> This 'if' statement seems a little magic. Does u_box_2d clamp
> width/height positive automatically? Is it possible to get negative
> width/height? If the answer is "yes; no" respectively, which seems to be
> how the code works, maybe add a quick comment explaining that.
Negative width or height isn't explicitly allowed in the
partial_update spec; by that omission I would say it's not defined
behaviour.
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] panfrost: Rewrite u-interleaving code
Hi Alyssa, On Tue, 25 Jun 2019 at 19:54, Alyssa Rosenzweig wrote: > @@ -2,6 +2,7 @@ > * Copyright (c) 2011-2013 Luc Verhaegen > * Copyright (c) 2018 Alyssa Rosenzweig > * Copyright (c) 2018 Vasily Khoruzhick > + * Copyright (c) 2019 Collabora Please use 'Collabora, Ltd.' as that's our full legal form. Afraid I have no feelings on the actual code however. ;) Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH v4 2/5] dri_interface: add DRI2_BufferDamage interface
On Tue, 25 Jun 2019 at 17:46, Boris Brezillon wrote: > On Tue, 25 Jun 2019 18:37:46 +0200 Boris Brezillon > wrote: > > Add a new DRI2_BufferDamage interface to support the > > EGL_KHR_partial_update extension, informing the driver of an overriding > > scissor region for a particular drawable. > > > > Based on a commit originally authored by: > > Harish Krupo > > renamed extension, retargeted at DRI drawable instead of context, > > rewritten description > > Oops, this patch is missing Daniel's SoB. Oops indeed! Signed-off-by: Daniel Stone Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] gbm: add gbm_{bo, surface}_create_with_modifiers2
Hi, On Tue, 25 Jun 2019 at 16:07, Jason Ekstrand wrote: > On Tue, Jun 25, 2019 at 4:04 AM Daniel Stone wrote: >> On Tue, 25 Jun 2019 at 07:26, Simon Ser wrote: >> > > I noticed that original patch (v1) for gbm_bo_create_with_modifiers did >> > > have usage at first but it was removed during the review. I'm having >> > > trouble digging what was the reason for this? >> > >> > I'm not sure either. Daniel said it was a mistake. >> > >> > Adding the 63bd2ae7452d4 folks to the discussion. Ben, do you remember >> > the details? >> >> We decided to remove it since we decided that modifiers were a good >> enough proxy for usage; no need to pass SCANOUT or TEXTURE anymore, >> because we already get the scanout modifiers from KMS and the texture >> modifiers from EGL. >> >> In hindsight, I think this was a mistake since it only handles buffer >> layout, and not buffer placement or cache configuration. > > > It's not great but modifiers should be able to handle that as well. You can > have _CONTIGUOUS versions of the modifiers supported by scanout and scanout > will only advertise those and the caller has to know to place them in > contiguous memory. That's just an example but I think it would probably work > for a lot of the cases. If not, I'd love to know why not. Sometimes it's _CONTIGUOUS, sometimes it's _ON_THIS_PCIE_DEVICE. Either way, it does seem like a bit of an abuse: it has nothing to do with internal buffer layout, but how and where the backing pages are sourced. Given that it's completely orthogonal, I wouldn't like to go trying to combine it into the same namespace. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] gbm: add gbm_{bo, surface}_create_with_modifiers2
Hi,
On Mon, 24 Jun 2019 at 19:51, Simon Ser wrote:
> +GBM_EXPORT struct gbm_bo *
> +gbm_bo_create_with_modifiers2(struct gbm_device *gbm,
> + uint32_t width, uint32_t height,
> + uint32_t format,
> + const uint64_t *modifiers,
> + const unsigned int count,
> + uint32_t usage)
> +{
> + if (width == 0 || height == 0) {
> + errno = EINVAL;
> + return NULL;
> + }
> +
> + if ((count && !modifiers) || (modifiers && !count)) {
> + errno = EINVAL;
> + return NULL;
> + }
> +
> + return gbm->bo_create(gbm, width, height, format, usage, modifiers,
> count);
This will trip an assert inside gbm_dri_bo_create:
/* Callers of this may specify a modifier, or a dri usage, but not both. The
* newer modifier interface deprecates the older usage flags.
*/
assert(!(usage && count));
You can fix this without modifying the libgbm <-> backend interface,
given that the two should be tightly coupled together. But within
Mesa, you'll need a new version of the DRIimageExtension, adding a
createImageWithModifiers2() which re-adds the flags, given that
they're not there in the current createImageWithModifiers() hook.
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] gbm: add gbm_{bo, surface}_create_with_modifiers2
Hi, On Tue, 25 Jun 2019 at 07:26, Simon Ser wrote: > > I noticed that original patch (v1) for gbm_bo_create_with_modifiers did > > have usage at first but it was removed during the review. I'm having > > trouble digging what was the reason for this? > > I'm not sure either. Daniel said it was a mistake. > > Adding the 63bd2ae7452d4 folks to the discussion. Ben, do you remember > the details? We decided to remove it since we decided that modifiers were a good enough proxy for usage; no need to pass SCANOUT or TEXTURE anymore, because we already get the scanout modifiers from KMS and the texture modifiers from EGL. In hindsight, I think this was a mistake since it only handles buffer layout, and not buffer placement or cache configuration. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Move adriconf to mesa repositories
Hi, On Tue, 2 Apr 2019 at 00:19, Rob Clark wrote: > On Mon, Apr 1, 2019 at 1:55 PM Jean Hertel wrote: > > As we have spoken already in the past, I have the intention to move > > adriconf under the mesa project umbrella, as an official tool for > > configuring DRI options. > > I would like to ask your advice, as well as other mesa developers, on how > > to properly do this. (If people is willing to accept it) > > So far the questions I have are: > > > > - What is the proccess to become an official mesa tool? > > Maybe a gitlab ticket? Or ping daniels and ask? > > > - Do I need any approval? Like from other mesa developers or X.Org > > Foundation? > > No approval from X.Org foundation or anything like that.. maybe some > general consensus among mesa dev's (or at least a few others who think > it is a good idea). For the record, I think living under > gitlab.freedesktop.org/mesa/adriconf makes sense. Right. As long as Mesa people agree then any project owner can just go ahead and create the owner in Mesa themselves; no need for admins to do anything. > > - Also, what about gitlab? If we move, can we use it? I already know the > > tool and would really appreciate using it. > > Yes, I don't think the fd.o admin's are creating new non-gitlab projects. Absolutely correct. > > - Is there anyone else willing to have commit rights on it? I known the > > project is public, but I feel it would be nice to have someone else also > > with commit/admin rights in case I'm hit by a bus :) > > Hmm, I guess it is possible to set it up so anyone with mesa commit > rights would have adriconf commit rights. But afaiu gitlab is > somewhat flexible on groups so we could I guess do something more fine > grained. It's quite fine-grained, yes. It inherits the permissions of the Mesa group, so anyone who has permission to Mesa can commit to it. You can also add committers specifically to the adriconf project, so they can only commit to that. Again this doesn't need admin intervention, you can just go ahead and do it yourself. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Mesa CI is too slow
Hi, On Mon, 18 Feb 2019 at 18:58, Eric Engestrom wrote: > On Monday, 2019-02-18 17:31:41 +0000, Daniel Stone wrote: > > Two hours of end-to-end pipeline time is also obviously far too long. > > Amongst other things, it practically precludes pre-merge CI: by the > > time your build has finished, someone will have pushed to the tree, so > > you need to start again. Even if we serialised it through a bot, that > > would limit us to pushing 12 changesets per day, which seems too low. > > > > I'm currently talking to two different hosts to try to get more > > sponsored time for CI runners. Those are both on hold this week due to > > travel / personal circumstances, but I'll hopefully find out more next > > week. Eric E filed an issue > > (https://gitlab.freedesktop.org/freedesktop/freedesktop/issues/120) to > > enable ccache cache but I don't see myself having the time to do it > > before next month. > > Just to chime in to this point, I also have an MR to enable ccache per > runner, which with our static runners setup is not much worse than the > shared cache: > https://gitlab.freedesktop.org/mesa/mesa/merge_requests/240 > > From my cursory testing, this should already cut the compilations by > 80-90% :) That's great! Is there any reason not to merge it? > > Doing the above would reduce the run time fairly substantially, for > > what I can tell is no loss in functional coverage, and bring the > > parallelism to a mere 1.5x oversubscription of the whole > > organisation's available job slots, from the current 2x. > > > > Any thoughts? > > Your suggestions all sound good, although I can't speak for #1 and #2. > > #3 sounds good, I guess we can keep meson builds with the "oldest supported > llvm" and the "current llvm version", and only the "oldest supported" > for autotools? We could have Meson building all the LLVM versions autotools does for not much overhead at all. At the moment though Meson builds 3 and autotools builds 6, which isn't bring us increased code coverage. > You've suggested reducing the amount that's built (ccache, > dropping/merging jobs) and making it more parallel (fewer jobs), but > there's another avenue to look at: run the CI less often. > > In my opinion, the CI should run on every single commit. Since that's > not realistic, we need to decide what's essential. > From most to least important: > > - master: everything that hits master needs to be build- and smoke-tested > > - stable branches: we obviously don't want to break stable branches > > - merge requests: the reason I wrote the CI was to automatically test MRs > > - personal work on forks: it would be really useful to test things > before sending out an MR, especially with the less-used build systems > that we often forget to update, but this should be opt-in, not opt-out > as it is right now. > > Ideally, this means we add this to the .gitlab.yml: > only: > - master > - merge_requests > - ci/* > > Until this morning, I thought `merge_requests` was an Enterprise Edition > only feature, which I why I didn't put it in, but it appears I was wrong, > see: > https://docs.gitlab.com/ce/ci/merge_request_pipelines/ > (Thanks Caio for reading through the docs more carefully than I did! :) > > I'll send an MR in a bit with the above. This will mean that master and > MRs get automatic CI, and pushes on forks don't (except the fork's > master), but one can push a `ci/*` branch to their own fork to run the > CI on it. > > I think this should massively drop the use of the CI, but mostly remove > unwanted uses :) It depends on the definition of 'unwanted', of course ... I personally like the idea of having a very early canary in the coalmine, and building it into peoples' workflows as quickly as possible. If a more sensible job split could reduce compilation time by 30-40%, and using ccache could drop the compilation overhead by a huge amount as well, that sounds like more than enough to not need to stop CI on personal forks. Why don't we pursue those avenues first, rather than restricting the audience? Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
[Mesa-dev] Mesa CI is too slow
Hi all, A few people have noted that Mesa's GitLab CI is just too slow, and not usable in day-to-day development, which is a massive shame. I looked into it a bit this morning, and also discussed it with Emil, though nothing in this is speaking for him. Taking one of the last runs as representative (nothing in it looks like an outlier to me, and 7min to build RadeonSI seems entirely reasonable): https://gitlab.freedesktop.org/mesa/mesa/pipelines/19692/builds This run executed 24 jobs, which is beyond the limit of our CI parallelism. As documented on https://www.freedesktop.org/wiki/Infrastructure/ we have 14 concurrent job slots (each with roughly 4 vCPUs). Those 24 jobs cumulatively took 177 minutes of execution time, taking 120 minutes for the end-to-end pipeline. 177 minutes of runtime is too long for the runners we have now: if it perfectly occupies all our runners it will take over 12 minutes, which means that even if no-one else was using the runners, they could execute 5 Mesa builds per hour at full occupancy. Unfortunately, VirGL, Wayland/Weston, libinput, X.Org, IGT, GStreamer, NetworkManager/ModemManager, Bolt, Poppler, etc, would all probably have something to say about that. When the runners aren't occupied and there's less contention for jobs, it looks quite good: https://gitlab.freedesktop.org/anholt/mesa/pipelines/19621/builds This run 'only' took 20.5 minutes to execute, but then again, 3 pipelines per hour isn't that great either. Two hours of end-to-end pipeline time is also obviously far too long. Amongst other things, it practically precludes pre-merge CI: by the time your build has finished, someone will have pushed to the tree, so you need to start again. Even if we serialised it through a bot, that would limit us to pushing 12 changesets per day, which seems too low. I'm currently talking to two different hosts to try to get more sponsored time for CI runners. Those are both on hold this week due to travel / personal circumstances, but I'll hopefully find out more next week. Eric E filed an issue (https://gitlab.freedesktop.org/freedesktop/freedesktop/issues/120) to enable ccache cache but I don't see myself having the time to do it before next month. In the meantime, it would be great to see how we could reduce the number of jobs Mesa runs for each pipeline. Given we're already exceeding the limits of parallelism, having so many independent jobs isn't reducing the end-to-end pipeline time, but instead just duplicating effort required to fetch and check out sources, cache (in the future), start the container, run meson or ./configure, and build any common files. I'm taking it as a given that at least three separate builds are required: autotools, Meson, and SCons. Fair enough. It's been suggested to me that SWR should remain separate, as it takes longer to build than the other drivers, and getting fast feedback is important, which is fair enough. Suggestion #1: merge scons-swr into scons-llvm. scons-nollvm will already provide fast feedback on if we've broken the SCons build, and the rest is pretty uninteresting, so merging scons-swr into scons-llvm might help cut down on duplication. Suggestion #2: merge the misc Gallium jobs together. Building gallium-radeonsi and gallium-st-other are both relatively quick. We could merge these into gallium-drivers-other for a very small increase in overall runtime for that job, and save ourselves probably about 10% of the overall build time here. Suggestion #3: don't build so much LLVM in autotools. The Meson clover-llvm builds take half the time the autotools builds do. Perhaps we should only build one LLVM variant within autotools (to test the autotools LLVM selection still works), and then build all the rest only in Meson. That would be good for another 15-20% reduction in overall pipeline run time. Suggestion #4 (if necessary): build SWR less frequently. Can we perhaps demote SWR to an 'only:' job which will only rebuild SWR if SWR itself or Gallium have changed? This would save a good chunk of runtime - again close to 10%. Doing the above would reduce the run time fairly substantially, for what I can tell is no loss in functional coverage, and bring the parallelism to a mere 1.5x oversubscription of the whole organisation's available job slots, from the current 2x. Any thoughts? Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
[Mesa-dev] Fwd: PSA: Mailman changes, From addresses no longer accurate
Hi all, Unfortunately, freedesktop.org's job of sending mail to a huge number of people whilst pretending to be other people, has just got even harder than it was. fd.o can no longer (at least for the moment) send mail with the From addresses of DMARC/DKIM/SPF-protected sender domains. When we try to do so, large providers reject the mail, despite DMARC records explicitly specifying that the mail should be accepted. Worse than that, not only does the immediate delivery attempt fail, but it punishes our sender reputation enough that _other_ mail delivery fails: after we hit a DMARC-related failure, large providers often refuse delivery attempts for mail from non-DMARC-protected domains. As a result, if the sender domain has a DMARC record, we rewrite the From address to be the list's address, preserving the original author in Reply-To. I'm chasing this up through a few channels, but in the meantime, please be aware that the From address on mails may no longer be accurate. If you are attempting to apply patches with git-am, please check that the commit author is not 'Person Name via listname-devel '. If you are replying privately to a list mail, please be _very_ careful that you are replying to the original sender (in Reply-To) and not the list itself. Cheers, Daniel -- Forwarded message - From: Daniel Stone Date: Mon, 11 Feb 2019 at 23:38 Subject: PSA: Mailman changes, From addresses no longer accurate To: , Hi all, We have hit another step change in aggressive anti-spam techniques from major mail providers. Over the past few days, we saw a huge spike in the number of mails we were failing to deliver to GMail and outlook.com in particular. It looks like it is now no longer acceptable for us to break DMARC/DKIM/SPF. These are DNS-based extensions to SMTP, which allow domains to publish policies as to who should be allowed to send email on their behalf. SPF provides source filtering, so e.g. freedesktop.org could specify that no-one should accept mail with a From: *@freedesktop.org unless it came from gabe.freedesktop.org. Mailman completely breaks this: if I specified a filter only allowing Google to send mail for @fooishbar.org, then anyone enforcing SPF would reject receipt of this mail, as it would arrive from fd.o with my From address. DKIM allows domains to publish a public key in DNS, inserting a header into mails sent from their domain cryptographically signing the value of named headers. Mailman breaks this too: changing the Sender header (such that bounces get processed by Mailman, rather than sending a deluge of out-of-office and mailbox-over-quota messages to whoever posts to the list) can break most DKIM signatures. Mailman adding the unsubscribe footer also breaks this; we could make it not add the footer, but in order to do so we'd have to convince ourselves that we were not decreasing our GDPR compliance. DMARC ties the two together, allowing domains to specify whether or not DKIM/SPF should be mandatory, and if they fail, what action should be taken. Despite some domains specifying a fail action of 'none' (receiving MTA to send an advisory report to a named email address, but still allow the email), it seems that popular services still interpret 'none' as 'reject'. As a result, Google in particular is dropping some number of our mails on the floor. This does _not_ just apply to mails which fail DMARC/DKIM/SPF: every mail we send that fails these filters (which is a lot - e.g. Intel and NVIDIA both have SPF) decreases our sender reputation with GMail and causes it to reject legitimate mails. I've reached out to Google through a couple of channels to see what we can do to increase our delivery rate to them. In the meantime, if your mail is hosted by Google, or Outlook, and you think you're missing mails - you probably are. Mailman has also now been reconfigured such that if it spots a potential DMARC violation, it rewrites the From address to be @lists.freedesktop.org, keeping the original author in Reply-To. It also strips DKIM headers. This seems to be about the best we can do, unless and until the major mail service providers offer us some alternate way to send mail. If you are replying privately to someone, you should check very carefully that you are replying to them and not to the list. Unfortunately we don't have a good answer in the long run. The latest published RFC at https://tools.ietf.org/html/rfc6377 suggests that there are no good solutions. If anyone is blessed with an abundance of time and familiar with the current email landscape, I would love to talk to you and get your help to work on this. Unfortunately we don't have the manpower to actually properly monitor email; it can often take a collapse in successful-delivery rates for us to notice. Ultimately, I suspect the best solution is to move most of our discussions to dedicated fora like GitLab issues, or something like Discourse. Fundamentally, the thing we're trying to do (send email
Re: [Mesa-dev] [PATCH 1/3] egl/android: Delete set_damage_region from egl dri vtbl
Hi, On Fri, 20 Jul 2018 at 19:32, Eric Anholt wrote: > Harish Krupo writes: > > Thank you, understood it. I should have read the spec better :(. > > Also, generalizing Android/deqp's usage seems to be wrong. Android's > > deqp passed previously even when the driver wasn't restricting the > > rendering to only the damaged regions. > > Should I update these in the comments section of the extension? > > Yes, please. If we're building an API, we should be documenting what it > does. I've resurrected this series in the form of a GitLab MR: https://gitlab.freedesktop.org/wayland/weston/merge_requests/106 Looking at the original implementation, I think it was misguided: the Intel driver hook replaced the scissor region in the context with the extents of the damage region. Unless I've completely misread the spec, the damage region should be additive to the scissor region, rather than replacing it: it should also only apply to winsys buffers and not user FBOs. Given that, I changed the DRI interface to target a drawable rather than a context, as well as cleaned it up, renamed it, added substantial documentation, and a Gallium + VC4 implementation. The MR is tagged WIP because I still have a lot of outstanding questions on how the implementation should look. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] freedreno: Fix meson build.
On Thu, 7 Feb 2019 at 14:59, Eric Engestrom wrote: > On Wednesday, 2019-02-06 18:36:09 +, Vinson Lee wrote: > > ../src/gallium/drivers/freedreno/freedreno_resource.c: In function > > ‘fd_resource_create_with_modifiers’: > > ../src/gallium/drivers/freedreno/freedreno_resource.c:884:30: error: > > ‘DRM_FORMAT_MOD_QCOM_COMPRESSED’ undeclared (first use in this function) > >allow_ubwc = find_modifier(DRM_FORMAT_MOD_QCOM_COMPRESSED, modifiers, > > count); > > That's a weird error... I would've expected the `#include ` > to fail with a "No such file". If you copied the wrong part of the error > message, can you update that? Presumably it just finds an outdated copy of drm_fourcc.h, which doesn't have the Qualcomm modifier, from the system include path. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH 13/13] wayland: Add buffer handling and visuals for fp16 formats
Hi Kevin,
On Mon, 28 Jan 2019 at 18:43, Kevin Strasser wrote:
> Set loader caps indicating that wayland can handle both rgba ordering and
> fp16 formats.
>
> NOTE: This requries libwayland to provide definitions for
> WL_SHM_FORMAT_XBGR16161616F and WL_SHM_FORMAT_ABGR16161616F
To be honest, we wouldn't be very likely to add those until at least
Pixman got support for FP16 formats. (Can you also upload those
through, e.g., TexImage2D when using GLES?)
> static const __DRIimageLoaderExtension image_loader_extension = {
> diff --git a/src/egl/wayland/wayland-drm/wayland-drm.c
> b/src/egl/wayland/wayland-drm/wayland-drm.c
> index 3c6696d..9dd7fd3 100644
> --- a/src/egl/wayland/wayland-drm/wayland-drm.c
> +++ b/src/egl/wayland/wayland-drm/wayland-drm.c
> @@ -117,6 +117,8 @@ drm_create_buffer(struct wl_client *client, struct
> wl_resource *resource,
> case WL_DRM_FORMAT_XRGB:
> case WL_DRM_FORMAT_YUYV:
> case WL_DRM_FORMAT_RGB565:
> +case WL_DRM_FORMAT_ABGR16161616F:
> +case WL_DRM_FORMAT_XBGR16161616F:
> break;
> default:
> wl_resource_post_error(resource,
> @@ -220,6 +222,10 @@ bind_drm(struct wl_client *client, void *data, uint32_t
> version, uint32_t id)
>WL_DRM_FORMAT_XRGB);
> wl_resource_post_event(resource, WL_DRM_FORMAT,
> WL_DRM_FORMAT_RGB565);
> +wl_resource_post_event(resource, WL_DRM_FORMAT,
> + WL_DRM_FORMAT_ABGR16161616F);
> +wl_resource_post_event(resource, WL_DRM_FORMAT,
> + WL_DRM_FORMAT_XBGR16161616F);
These should only be advertised if the underlying driver actually
supports texturing from FP16.
Regardless, we try to avoid adding ~anything new to wl_drm. It would
be much better to have DRM FourCC definitions for these formats and
use them via the linux_dmabuf extension if available.
If the driver advertises DRM_FORMAT_*16161616F formats through the EGL
dmabuf format query extension, then the server will automatically
advertise them to clients and import will just work on the server side
without having to touch src/egl/wayland/wayland-drm/, which we'd
mostly prefer to be preserved in amber.
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Fwd: Review for last remaining Mesa wayland-drm depth 30 bits?
Hi, On Tue, 29 Jan 2019 at 16:05, Adam Jackson wrote: > On Tue, 2019-01-29 at 14:45 +0100, Mario Kleiner wrote: > > could i get some review of the last two missing patches of mine for > > depth 30 support in Mesa's egl/wayland wl-drm backend? They are over > > six months old now, well-tested at time of original submission: > > > > https://patchwork.freedesktop.org/project/mesa/list/?submitter=14956 > > > > Would be good to get this into Mesa 19 before new color formats are > > added. Should be useful for new formats as well. > > 4 and 5 are: > > Reviewed-by: Adam Jackson And they are also: Reviewed-by: Daniel Stone Thanks for chasing this up! Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Thoughts after hitting 100 merge requests?
Hi, On Fri, 25 Jan 2019 at 23:24, Rob Clark wrote: > (Hmm, I guess I should take a look at what sort of API gitlab offers, > but that will probably have to wait until after the branchpoint.. tbh > I'd actually be pretty happy w/ a gitlab equiv of 'git pw as -s' for > merging things from cmdline.) Add this to your Git remote specification: fetch = +refs/merge-requests/*:refs/heads/mr/* And then the actual branch submitted for an MR will be present as mr/100/head. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] docs: add note about sending merge-requests from forks
Hi, On Wed, 23 Jan 2019 at 10:09, Eric Engestrom wrote: > On Wednesday, 2019-01-23 10:36:15 +0100, Erik Faye-Lund wrote: > > Sending MRs from the main Mesa repository increase clutter in the > > repository, and decrease visibility of project-wide branches. So it's > > better if MRs are sent from forks instead. > > > > Let's add a note about this, in case its not obvious to everyone. > > Yes please, we already have way too many dead branches in the main repo > without adding that kind of things to it. > > One other thing is that (last time I checked) the scripts propagating > the repo to github et al. don't propagate branch deletions, which means > the clutter never gets cleaned there. They're propagated from gitlab.fd.o to git.fd.o, as the hook is run within regular post-receive, but you're right that pushing from git.fd.o to GitHub doesn't notice old branches, as it just pushes everything present in the git.fd.o repo to GitHub, and doesn't notice any branches no longer present. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Thoughts after hitting 100 merge requests?
Hi, On Thu, 17 Jan 2019 at 16:35, Jason Ekstrand wrote: > On January 17, 2019 08:58:03 Erik Faye-Lund > wrote: > > Whoops! I meant to say something like "we'd need to be able to > > distinguis between CI steps that are triggered due to new MRs versus > > updated MRs, or pushes to existing branches". > > > >> Anyway, Jason did actually write that hook, and it's something I'm > >> happy to host on existing fd.o machines. I just haven't got to doing > >> it, since I ended up taking my sabbatical a lot more seriously than I > >> expected, and now I'm back to work I've got a bit of a backlog. But > >> we > >> can definitely do it, and pretty soon. > > > > Cool, then I won't worry about it, and just assume it'll appear > > magically soon :) > > My script was a total hack. It's probably massively insecure and doesn't > include any code to provide a diffstat which has been requested by several > people. Someone taking it a bit more seriously would probably be good > before we deploy anything. With the caveat that I can no longer see the script because it's been expired out of the pastebin (why not make a GitLab repo or at least upload it to a snippet?) ... I had the same assumption when you posted it, but came to the conclusion it was actually OK, or at least would be with very minimal work. We can configure Apache and GitLab pretty easily so it can only be triggered with a secret token which is buried in the repo config and/or accessible only to admins. It calls back into GitLab to get the changes, so there's no danger of it sending completely arbitrary content even if someone does figure out how to trigger it when they shouldn't. It also has GitLab project -> email destination hardcoded in the script, so there's no danger of it being used to spam arbitrary addresses either. Even without that, given that people currently only need to sign up to Bugzilla (without a captcha) in order to send email with arbitrary content to mesa-dev@, 'less spam-prone than the status quo' is an embarrassingly low bar anyway. Whoever wants to see this happen should ping Jason to get the script and his suggested changes, get it in a GitLab repo, then file an issue on https://gitlab.freedesktop.org/freedesktop/freedesktop/issues/new/ and I'll get it deployed. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Thoughts after hitting 100 merge requests?
Hi, On Thu, 17 Jan 2019 at 07:38, Erik Faye-Lund wrote: > 1. New MRs should probably get their cover-letter automatically sent to > the mailing list for incrased visibility. > > [...] > > I don't think any of these issues are show-stoppers from moving > entirely to MRs, though. Perhaps issue #1 here should be fixed first, > dunno... > > Issue #1 is of course "just" a matter of writing some script that gets > triggered by a hook. It's just work that needs to be done; any of us > that complain about it could take up the work, I suppose. > > The only question is where to host such a service... perhaps it's just > a gitlab CI step? That way we'd get some "automatic" cloud computing > power to do the work, but it doesn't really seem like a perfect fit; > we'd need to be able to distinguis between CI steps that are triggered > due to new Unfinished sentence? Anyway, Jason did actually write that hook, and it's something I'm happy to host on existing fd.o machines. I just haven't got to doing it, since I ended up taking my sabbatical a lot more seriously than I expected, and now I'm back to work I've got a bit of a backlog. But we can definitely do it, and pretty soon. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Thoughts after hitting 100 merge requests?
Hi, On Wed, 16 Jan 2019 at 13:01, Lionel Landwerlin wrote: > - It seems we only get notifications when adding to an MR, I could like to > subscribe to particular tags If you go to https://gitlab.freedesktop.org/mesa/mesa/labels/ then you can subscribe to things per-label. That applies to both issues and MRs, but might help. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Thoughts after hitting 100 merge requests?
Hi,
On Tue, 15 Jan 2019 at 23:47, Matt Turner wrote:
> On Mon, Jan 14, 2019 at 4:36 AM Daniel Stone wrote:
> > On Fri, 11 Jan 2019 at 17:05, Jason Ekstrand wrote:
> > > 5. There's no way with gitlab for Reviewed-by tags to get automatically
> > > applied as part of the merging process. This makes merging a bit more
> > > manual than it needs to be but is really no worse than it was before.
> >
> > I'm still on the side of not seeing the value in them.
>
> Reviewed-by tags are useful for measuring the quantity of patch review
> people do (which is useful in a corporate environment...). It's often
> a thankless task that's valued much lower than first order
> contributions, so having a way to at least quantify patch reviews
> shows that people are spending their time to help others contribute.
>
> The number of R-b tags is not a 100% accurate picture of the
> situation, but it gives at least a good overview of who is doing the
> tedious work of patch review. For instance, in 2018 the top reviewers
> are
>
> 620 Bas Nieuwenhuizen
> 530 Marek Olšák
> 505 Jason Ekstrand
> 452 Kenneth Graunke
>
> If my name were in there, it would definitely be something I put on my
> yearly review.
Obviously shell scripting will not do in the advanced enterprise
environment, so we're into the realm of Python. Nevertheless, I've
attached a script which will let you prove your individual worth to
the company, as well as producing a leaderboard for team morale
purposes:
~/tmp % ./kwg-gitlab-rb.py
kwg participated in 10 merged MRs this year (61 total):
MR 105: gallium: Add a PIPE_CAP_QUERY_PIPELINE_STATISTICS_SINGLE
capability and query type
MR 96: st/mesa: Optionally override RGB/RGBX dst alpha blend factors
MR 95: st/nir: Lower TES gl_PatchVerticesIn to a constant if
linked with a TCS.
MR 92: nir: Allow a non-existent sampler deref in
nir_lower_samplers_as_deref
MR 62: i965: fix VF cache issue workaround
MR 39: i965: Enable software support for
ARB_gpu_shader_fp64/ARB_gpu_shader_int64
MR 26: Revert "nir/lower_indirect: Bail early if modes == 0"
MR 12: Add preemption support to i965 on gen9+
MR9: Misc optimizations
MR4: mesa/st: Expose compute shaders when NIR support is advertised.
Top 10 MR participants:
29 jekstrand
16 anholt
14 bnieuwenhuizen
10 jljusten
10 kwg
10 idr
9 dbaker
9 eric
8 llandwerlin
6 cmarcelo
Cheers,
Daniel
#!/usr/bin/python3
from collections import Counter
import datetime
from itertools import chain
import gitlab
import os
import sys
MY_USERNAME="kwg"
LEADERBOARD_LEN=10
# Look back over the past year
end_of_year = datetime.datetime.now()
start_of_year = end_of_year - datetime.timedelta(days=365)
# ... or calendar years are available
#start_of_year = datetime.datetime(year=2017, month=12, day=31)
#end_of_year = datetime.datetime(year=2019, month=1, day=1)
gl = gitlab.Gitlab("https://gitlab.freedesktop.org;, os.environ["GITLAB_TOKEN"], api_version=4)
project = gl.projects.get(id="mesa/mesa")
merged_mrs = project.mergerequests.list(state="merged", updated_after=start_of_year, updated_before=end_of_year, all=True, as_list=False)
my_mrs = list(filter(lambda mr: MY_USERNAME in [p["username"] for p in mr.participants()], merged_mrs))
print("{user} participated in {me} merged MRs this year ({all} total):".format(user=MY_USERNAME, me=len(my_mrs), all=len(list(merged_mrs
for mr in my_mrs:
print("MR {:4d}: {}".format(mr.iid, mr.title))
print()
print("Top {} MR participants:".format(LEADERBOARD_LEN))
leaderboard = Counter(chain.from_iterable(map(lambda mr: [p["username"] for p in mr.participants()], merged_mrs)))
for hero, count in leaderboard.most_common(LEADERBOARD_LEN):
print("{:3d} {}".format(count, hero))
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Thoughts after hitting 100 merge requests?
Hey,
On Tue, 15 Jan 2019 at 20:22, Rob Clark wrote:
> On Tue, Jan 15, 2019 at 7:40 AM Daniel Stone wrote:
> > My question would again be what value that brings you. Do you just
> > like seeing the name there, or do you go poke the people on IRC, or
> > follow up via email, or ... ? Again I personally go look through the
> > original review to see what came up during that first, but everyone's
> > different, so I'm just trying to understand what you actually do with
> > that information, so we can figure out if there's a better way to do
> > things for everyone rather than just blindly imitating what came
> > before.
>
> If I am curious or have some questions about why some code is the way
> it is I frequently use tig-blame, which makes it easy to step into the
> commit that made the change and see the commit msg and r-b tags.. I
> guess the most important part if I need to ping someone on IRC w/
> questions is the author, but it seems like having the other tags handy
> without context-switching to browser/gitlab is useful.
>
> I guess I don't as frequently dig into the history of the original
> patchset and it's review comments.. mostly because that isn't as easy
> with the email based review process. Making this easier would defn be
> a win. But in cases where I don't have to leave the comfort of tig,
> it would be nice not to have to start doing so..
>
> This is not an argument for sticking to email based process, just
> defence of what I think would be a useful feature for gitlab to gain
> ;-)
Thanks, that helps. How about this? It technically even fits in one
line, though you might wish it didn't.
~/mesa/mesa master ← → * % export
GITLAB_TOKEN=secret-api-token-you-get-from-web-UI
~/mesa/mesa master ← → * % export
GITLAB_COMMIT=f967273fb442de8281f8248e8c8bff5b13ab89e4
~/mesa/mesa master ← → * % curl --silent --header "PRIVATE-TOKEN:
$GITLAB_TOKEN"
https://gitlab.freedesktop.org/api/v4/projects/mesa%2Fmesa/merge_requests/$(curl
--silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN"
https://gitlab.freedesktop.org/api/v4/projects/mesa%2Fmesa/repository/commits/${GITLAB_COMMIT}/merge_requests
| jq -r '.[] | .iid')/participants | jq -r '.[] | { username:
.username, realname: .name }'
{
"username": "sroland",
"realname": "Roland Scheidegger"
}
{
"username": "kwg",
"realname": "Kenneth Graunke"
}
{
"username": "mareko",
"realname": "Marek Olšák"
}
{
"username": "tpalli",
"realname": "Tapani Pälli"
}
> (Also, I suppose preserving those artifacts of "the old process" is
> probably useful for folks who run git statistics, although personally
> that does not effect me.)
[mumbles something about GDPR]
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] egl/wayland: break double/tripple buffering feedback loops
Hi, On Tue, 18 Dec 2018 at 17:59, Lucas Stach wrote: > Am Dienstag, den 18.12.2018, 17:43 + schrieb Emil Velikov: > > > On Tue, 18 Dec 2018 at 11:16, Lucas Stach wrote: > > > if (dri2_surf->back == NULL) > > > dri2_surf->back = _surf->color_buffers[i]; > > > - else if (dri2_surf->back->dri_image == NULL) > > > + else if (dri2_surf->back->dri_image == NULL && > > > dri2_surf->color_buffers[i].dri_image) > > > dri2_surf->back = _surf->color_buffers[i]; > > > + age = dri2_surf->back->age; > > >} > > > > > > > AFAICT this is the wayland equivalent of > > 4f1d27a406478d405eac6f9894ccc46a80034adb > > Where the exact same logic/commit message applies. > > No it isn't. It's exactly what it says in the commit log. It's keeping > the tripple buffer around for a bit, even if we don't strictly need it > when the client is currently doing double buffering. Right - the crucial part in Derek's GBM commit was removing the 'break' and adding the extra conditional on age. Derek's patch stabilises the age of buffers handed back to the user, by always returning the oldest available buffer. That slightly pessimises rendering if there is a 'free' buffer in the queue: if four buffers are allocated, then we will always return a buffer from three flips ago, maybe meaning more rendering work. It means that, barring the client holding on to one buffer for unexpectedly long, the age of the oldest buffer in the queue will never be greater than the queue depth. This patch instead relies on unbalanced ages, where older buffers in the queue are allowed to age far beyond the queue depth if not used during normal rendering. > When things are on the edge between double buffering being enough and > sometimes a third buffer being needed to avoid stalling we would > otherwise bounce rapidly between allocating and disposing the third > buffer. > > The DRM platform has no such optimization and just keeps the third > buffer around forever. This patch keeps the optimization in the Wayland > platform, but adds a bit of hysteresis before disposing the buffer when > going from tripple to double buffering to see if things are settling on > double buffering. Ideally we'd have globally optimal behaviour for both platforms, but that doesn't really seem doable for now. I think this is a good balance though. There will only be one GBM user at a time, so having that allocate excessive buffers doesn't seem too bad, and the penalty for doing so is your entire system stuttering as the compositor becomes blocked. Given the general stability of compositors, if they need a larger queue depth at some point, they are likely to need it again in the near future. Conversely, there may be a great many Wayland clients, and these clients may bounce between overlay and GPU composition. Given that, it seems reasonable to opportunistically free up buffers, to make sure we have enough memory available across the system. > > The age check here seems strange - both number used and it's relation > > to double/triple buffering. > > Have you considered tracking/checking how many buffers we have? > > A hysteresis value of 18 is just something that worked well in > practice. It didn't appear to defer the buffer destruction for too long > while keeping the feedback loop well under control. Yeah, having this #defined with a comment above it would be nice. With that, this patch is: Reviewed-by: Daniel Stone Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Thoughts after hitting 100 merge requests?
Hi, On Tue, 15 Jan 2019 at 12:21, Rob Clark wrote: > On Tue, Jan 15, 2019 at 1:02 AM Tapani Pälli wrote: > > On 1/14/19 2:36 PM, Daniel Stone wrote: > > > On Fri, 11 Jan 2019 at 17:05, Jason Ekstrand wrote: > > > In other projects, we looked for ways to apply the tags and ended up > > > concluding that they didn't bring enough value to make it worthwhile. > > > I don't know if that holds for Mesa, but it would be better to start > > > with an actual problem statement - what value does R-b bring and how? > > > - then look at ways to solve that problem, rather than just very > > > directly finding a way to insert that literal text string into every > > > commit message. > > > > IMO it brings some 'shared responsibility' for correctness of the patch Oh, no doubt - we certainly haven't abandoned thorough review! So far we haven't seen that compromised by not having a name in the commit message. > > and quickly accessible information on who were looking at the change. So > > ideally later when filing bug against commit/series there would be more > > people than just the committer that should take a look at the possible > > regressions. At least in my experience people filing bugs tend to often > > also CC the reviewer. Yeah, that's really helpful. So maybe a useful flow - assuming we eventually switch to GitLab issues - would be the ability to associate an issue with a commit, which could then automatically drag in people who commented on the MR which landed that commit, as well as (at least) the reporter of the issue(s) fixed by that MR. That would need some kind of clever - probably at least semi-manual - filtering to make sure it wasn't just spamming the world, but it's at least a starting point. > +1 .. and also it is nice to see things like Reported-by/Reviewed-by > without having to go search somewhere else (ie. outside of git/tig) My question would again be what value that brings you. Do you just like seeing the name there, or do you go poke the people on IRC, or follow up via email, or ... ? Again I personally go look through the original review to see what came up during that first, but everyone's different, so I'm just trying to understand what you actually do with that information, so we can figure out if there's a better way to do things for everyone rather than just blindly imitating what came before. > (ofc it would be pretty awesome incentive to switch to gitlab issues > if gitlab could automate adding Reported-by tags for MR's associated > with an issue.. but I guess checkbox to add Reviewed-by tag would > already make my day) I saw this the other day, which might be more incentive: https://csoriano.pages.gitlab.gnome.org/csoriano-blog/post/2019-01-07-issue-handling-automation/ Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Thoughts after hitting 100 merge requests?
Hi, On Fri, 11 Jan 2019 at 17:05, Jason Ekstrand wrote: > 5. There's no way with gitlab for Reviewed-by tags to get automatically > applied as part of the merging process. This makes merging a bit more manual > than it needs to be but is really no worse than it was before. I'm still on the side of not seeing the value in them. Most of the time when I go to pursue someone who reviewed a commit, I'll go to see what came up in review anyway. Maybe someone had the same comment which was found to be not applicable or otherwise explained away. Reviewed-by and Acked-by are also pretty lossy anyway, and freeform text descriptors in a comment can much better capture the intent (e.g. 'I'm strongly OK with the driver changes and weakly OK with the core changes as it's not really my area of expertise'). In other projects, we looked for ways to apply the tags and ended up concluding that they didn't bring enough value to make it worthwhile. I don't know if that holds for Mesa, but it would be better to start with an actual problem statement - what value does R-b bring and how? - then look at ways to solve that problem, rather than just very directly finding a way to insert that literal text string into every commit message. FWIW, if you go to https://gitlab.freedesktop.org/mesa/mesa/commit/SHA1 then you get a hyperlink from the web UI which points you to the MR. The API to do this is pretty straightforward and amenable to piping through jq: https://docs.gitlab.com/ce/api/commits.html#list-merge-requests-associated-with-a-commit Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH v2] docs: Document GitLab merge request process (email alternative)
Hi, On Sat, 8 Dec 2018 at 05:15, Eric Engestrom wrote: > On Friday, 2018-12-07 10:19:23 +0100, Erik Faye-Lund wrote: > > Automated emails (and perhaps IRC bot) would be really nice. > > Agreed. Email would be great to help with the transition. > There's work currently being done on GitLab to allow for mailing lists > to be notified; this should cover 'new MR' as well. > If we need this feature before GitLab is done, it should be possible to > write a bot using the webhooks, just needs someone to take the time to > do it :) > > For IRC, there's already some integration, but it's limited to notifying > about git pushes for now: > https://docs.gitlab.com/ee/user/project/integrations/irker.html > > There's an open issue about adding more events, but it hasn't seen much > activity: > https://gitlab.com/gitlab-org/gitlab-ce/issues/7965 Wayland uses a couple of eventd plugins chained together: https://github.com/sardemff7/git-eventc That notifies the channel when issues and MRs are opened or closed and on push as well, including things like the labels. It's been pretty useful so far. > > Even better if it could be hooked up to scripts/get_reviewer.pl, and > > automatically CC "the right people". > > Side note, I've been rewriting that script, although I need to send v2 > out at some point: > https://patchwork.freedesktop.org/patch/226256/ > > I would be trivial to hook that into a bot we'd write, but I don't think > GitLab has support for something like this. I just opened an issue about > adding support directly in GitLab: > https://gitlab.com/gitlab-org/gitlab-ce/issues/55035 This already exists, as an EE-only feature called 'code owners': https://docs.gitlab.com/ee/user/project/code_owners.html https://gitlab.com/gitlab-org/gitlab-ee/issues/1012 Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] docs: Document optional GitLab code review process
Hi all, Thanks for the CC. I'm on a sabbatical until mid-January; I'll be around but not following the lists/etc as actively as before. Please feel free to liberally CC me (on this address, not work) or poke me on IRC if there's something I should see or could contribute to. I'll have limited time for the next week or so, but should be able to do more on it next month, albeit in Australian timezone. On Wed, 28 Nov 2018 at 17:23, Dylan Baker wrote: > Quoting Matt Turner (2018-11-27 19:20:09) > > Discussion point: I think attempting to have simultaneous review in > > two places is a recipe for wasted time. Strawman: maybe we can only > > email the cover letter to the mailing list and include in it a link to > > the MR? > > I think it's a *really* bad idea to allow both. Some people will immediately > begin using MR/PR's only and never read the list, others will never check > MR/PRs and only use the list. It'll leave the rest of us forced to use both. > We > should either go all in and archive the mailing list and use only gitlab, or > not > use PR/MR's IMHO. I fully agree, for all the reasons listed here. It imposes a much higher overhead on contributors, and means that reviewers effectively need to work their way through both interfaces anyway. Since you don't get stable links until you push, in order to link between the two, you would need to submit a MR to GitLab, capture the URL, include that in your cover letter / patches, git-send-email, then capture the message ID / archive link and go back to GitLab to edit your cover letter and insert the link. Not to mention that it would basically destroy the usefulness of Patchwork overnight. Weston has a pretty conservative development workflow, but even then we've had near-zero pushback, and saw a pretty big leap in contributions from people who have been sitting on patchsets for a long time but not bothered to submit them when we had review on the list. I was genuinely surprised at how big the uptick was. > > > + Never use the merge button on the GitLab page to push patches > > > > Can we disable this in Gitlab? If the button is there, people *will* > > accidentally press it. > > I think Daniel was working on getting a "rebase and push" button like github > has > so that we could use the button, CC'd. This is already there, and can be configured per-repo. Under settings -> merge requests, choose this as your merge strategy: 'Fast-forward merge No merge commits are created and all merges are fast-forwarded, which means that merging is only allowed if the branch could be fast-forwarded. When fast-forward merge is not possible, the user is given the option to rebase.' If the user selects the 'allow commits from members who can merge to the target branch' checkbox when submitting the MR, reviewers can themselves request a rebase. We have FF-only MRs in Weston, and when I want to land a contribution after review, I hit rebase followed by 'merge after pipeline succeeds', which does what it says on the box, i.e. waits until CI has finished and then automatically lands it. There are a couple of rough edges here. First, users have to explicitly select the checkbox for this to happen; an upstream feature request to make this the default or even mandatory is at https://gitlab.com/gitlab-org/gitlab-ce/issues/49323. Secondly, if something else lands in the target branch between the rebase starting and CI finishing (i.e. it's no longer an FF landing by the time CI reports success), the reviewer has to request the rebase again. Jason was discussing having GitLab busy-loop, running rebase+CI in a loop until it could actually land. I don't have a link to hand for this though. > > I'm not sure if it's a custom thing or what (I can find out) but I'd > > much prefer to automate things if we can. Just like patchwork, people > > *will* forget to close merge requests if it's possible. (And people > > currently *do* forget to close bugzilla bugs) > > Or we could just use gitlab's issue tracker, which will automatically close > issues when a commit pushed to master has a tag like `Fixes: #1234` or > `Closes: #1234`. Right, and obviously if you land a MR through the web UI then it's automatically closed. > Personally speaking, I think that better next steps for gitlab integration > are: > - migrate from bugzilla to gitlab issues This is currently held up by a mutual death grip: both AMD and Intel want to be able to move or reassign issues between kernel / Mesa / X11 within Bugzilla, so have requested that nothing moves until everything moves. I don't know whether that has to be one big flag day or whether the driver teams would find it acceptable for all three components to have a documented plan with a timeline on it. Intel also have some pretty heavy scripting and tooling around Bugzilla which would need to be modified to track GitLab instead. From an infrastructure point of view though, Bugzilla is getting less and less acceptable to run as a service. It's no
Re: [Mesa-dev] [Mesa-stable] [PATCH] vulkan/wsi/wayland: Respect non-blocking AcquireNextImage
Hi Emil, On Thu, 8 Nov 2018 at 15:26, Emil Velikov wrote: > On Tue, 30 Oct 2018 at 12:57, Daniel Stone wrote: > > If the client has requested that AcquireNextImage not block at all, with > > a timeout of 0, then don't make any non-blocking calls. > > > > This will still potentially block infinitely given a non-infinte > > timeout, but the fix for that is much more involved. > > Did this fall through the cracks or there's another patch that supersedes it? To be honest, I'm slightly apprehensive about merging it, given the open questions in https://bugs.freedesktop.org/show_bug.cgi?id=108540#c1 Getting some feedback from others would be great. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH 3/3] egl: Improve the debugging of gbm format matching in DRI configs.
On Thu, 8 Nov 2018 at 18:01, Eric Anholt wrote: > Previously the debug would be: > > libEGL debug: No DRI config supports native format 0x20203852 > libEGL debug: No DRI config supports native format 0x38385247 > > but > > libEGL debug: No DRI config supports native format R8 > libEGL debug: No DRI config supports native format GR88 > > is a lot easier to understand. Series is: Reviewed-by: Daniel Stone ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH 1/2] gbm: Introduce a helper function for printing GBM format names.
Hi,
On Tue, 6 Nov 2018 at 13:11, Eric Engestrom wrote:
> On Friday, 2018-11-02 14:40:49 -0700, Eric Anholt wrote:
> > +GBM_EXPORT char *
> > +gbm_format_get_name(uint32_t gbm_format, struct gbm_format_name_desc *desc)
> > +{
>
> Actually, This won't work with the two GBM_BO_FORMAT_{X,A}RGB; you
> should add them as a special case here, something like:
>
> if (gbm_format == GBM_BO_FORMAT_XRGB)
> return "XRGB";
>
> if (gbm_format == GBM_BO_FORMAT_ARGB)
> return "ARGB";
>
> followed by your desc->name code.
Perhaps better would be to just lift gbm_format_canonicalize() from
src/gbm/backends/dri/gbm_dri.c and use it at function entry.
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] docs: mention EXT_shader_implicit_conversions
On Fri, 2 Nov 2018 at 15:58, Emil Velikov wrote: > Ff| 2 ++ > docs/relnotes/19.0.0.html | 2 +- > 2 files changed, 3 insertions(+), 1 deletion(-) > create mode 100644 Ff > > diff --git a/Ff b/Ff > new file mode 100644 > index 000..804e31ab99e > --- /dev/null > +++ b/Ff > @@ -0,0 +1,2 @@ > +sdasdads > + ... ? Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
[Mesa-dev] [PATCH] gbm: Clarify acceptable formats for gbm_bo
gbm_bo_create() was presumably meant to originally accept gbm_bo_format enums, but it's accepted GBM_FORMAT_* tokens since the dawn of time. This is good, since gbm_bo_format is rarely used and covers a lot less ground than GBM_FORMAT_*. Change the documentation to refer to both; this involves removing a 'see also' for gbm_bo_format, since we can't also use \sa to refer to a family of anonymous #defines. Signed-off-by: Daniel Stone Reported-by: Pekka Paalanen --- src/gbm/main/gbm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/gbm/main/gbm.c b/src/gbm/main/gbm.c index 0bf2922bacd..ee4fd3bd444 100644 --- a/src/gbm/main/gbm.c +++ b/src/gbm/main/gbm.c @@ -448,14 +448,14 @@ gbm_bo_destroy(struct gbm_bo *bo) * \param gbm The gbm device returned from gbm_create_device() * \param width The width for the buffer * \param height The height for the buffer - * \param format The format to use for the buffer + * \param format The format to use for the buffer, from GBM_FORMAT_* or + * GBM_BO_FORMAT_* tokens * \param usage The union of the usage flags for this buffer * * \return A newly allocated buffer that should be freed with gbm_bo_destroy() * when no longer needed. If an error occurs during allocation %NULL will be * returned and errno set. * - * \sa enum gbm_bo_format for the list of formats * \sa enum gbm_bo_flags for the list of usage flags */ GBM_EXPORT struct gbm_bo * -- 2.19.1 ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
[Mesa-dev] [PATCH] vulkan/wsi/wayland: Respect non-blocking AcquireNextImage
If the client has requested that AcquireNextImage not block at all, with a timeout of 0, then don't make any non-blocking calls. This will still potentially block infinitely given a non-infinte timeout, but the fix for that is much more involved. Signed-off-by: Daniel Stone Cc: mesa-sta...@lists.freedesktop.org Cc: Chad Versace Cc: Jason Ekstrand Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108540 --- src/vulkan/wsi/wsi_common_wayland.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/vulkan/wsi/wsi_common_wayland.c b/src/vulkan/wsi/wsi_common_wayland.c index e9cc22ec603..981243d8b14 100644 --- a/src/vulkan/wsi/wsi_common_wayland.c +++ b/src/vulkan/wsi/wsi_common_wayland.c @@ -699,9 +699,14 @@ wsi_wl_swapchain_acquire_next_image(struct wsi_swapchain *wsi_chain, } } - /* This time we do a blocking dispatch because we can't go - * anywhere until we get an event. + /* We now have to do a blocking dispatch, because all our images + * are in use and we cannot return one until the server does. However, + * if the client has requested non-blocking ANI, then we tell it up front + * that we have nothing to return. */ + if (info->timeout == 0) + return VK_NOT_READY; + int ret = wl_display_roundtrip_queue(chain->display->wl_display, chain->display->queue); if (ret < 0) -- 2.19.1 ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Allow fd.o to join forces with X.Org
Hi, On Fri, 26 Oct 2018 at 11:57, Daniel Vetter wrote: > On Fri, Oct 26, 2018 at 10:13:51AM +1000, Peter Hutterer wrote: > > On Wed, Oct 17, 2018 at 02:37:25PM +0200, Daniel Vetter wrote: > > > On Wed, Oct 17, 2018 at 2:05 PM Daniel Stone wrote: > > > > Yeah, I think it makes sense. Some things we do: > > > > - provide hosted network services for collaborative development, > > > > testing, and discussion, of open-source projects > > > > - administer, improve, and extend this suite of services as necessary > > > > - assist open-source projects in their use of these services > > > > - purchase, lease, or subscribe to, computing and networking > > > > infrastructure allowing these services to be run > > > > > > I fully agree that we should document all this. I don't think the > > > bylaws are the right place though, much better to put that into > > > policies that the board approves and which can be adapted as needed. > > > Imo bylaws should cover the high-level mission and procedural details, > > > as our "constitution", with the really high acceptance criteria of > > > 2/3rd of all members approving any changes. Some of the early > > > discussions tried to spell out a lot of the fd.o policies in bylaw > > > changes, but then we realized it's all there already. All the details > > > are much better served in policies enacted by the board, like we do > > > with everything else. > > > > > > As an example, let's look at XDC. Definitely one of the biggest things > > > the foundation does, with handling finances, travel sponsoring grants, > > > papers committee, and acquiring lots of sponsors. None of this is > > > spelled out in the bylaws, it's all in policies that the board > > > deliberates and approves. I think this same approach will also work > > > well for fd.o. > > > > > > And if members are unhappy with what the board does, they can fix in > > > the next election by throwing out the unwanted directors. > > > > yeah, fair call. though IMO in that case we can just reduce to > > > >\item Support free and open source projects through the freedesktop.org > >infrastructure. > > > > because my gripe is less with the fdo bit but more with defining what > > "project hosting" means, given that we use that term to exclude fdo projects > > from getting anything else. I think just dropping that bit is sufficient. > > Hm yeah, through the lens of "everything not explicitly listed isn't in > scope as X.org's purpose", leaving this out is probably clearest. And > under 2.4 (i) the board already has the duty to interpret what exactly > this means wrt membership eligibility. > > Harry, Daniel, what do you think? Yeah, that's fine. I didn't specifically want the enumerated list of what we do in the bylaws, just spelling it out for background as a handy reference I could point to later. I think maybe we could reduce it to something like: Administer, support, and improve the freedesktop.org hosting infrastructure to support the projects it hosts Gives us enough scope to grow in the future (e.g. we don't need a bylaws change to move from pure-git to GitLab), avoids the sticky question of what exactly fd.o hosts in the bylaws (e.g. if NetworkManager needs a new repo then we don't have to consult membership to add it), but is still pretty firmly limited in scope. Any of the above have my in-principle ack though, I think they're all reasonable colours for our lovely shed. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [RFC] Allow fd.o to join forces with X.Org
On Tue, 16 Oct 2018 at 08:17, Peter Hutterer wrote:
> On Mon, Oct 15, 2018 at 10:49:24AM -0400, Harry Wentland wrote:
> > + \item Support free and open source projects through the
> > freedesktop.org
> > + infrastructure. For projects outside the scope of item (\ref{1})
> > support
> > + extends to project hosting only.
> > +
>
> Yes to the idea but given that the remaining 11 pages cover all the legalese
> for xorg I think we need to add at least a section of what "project hosting"
> means. Even if it's just a "includes but is not limited to blah". And some
> addition to 4.1 Powers is needed to spell out what the BoD can do in regards
> to fdo.
Yeah, I think it makes sense. Some things we do:
- provide hosted network services for collaborative development,
testing, and discussion, of open-source projects
- administer, improve, and extend this suite of services as necessary
- assist open-source projects in their use of these services
- purchase, lease, or subscribe to, computing and networking
infrastructure allowing these services to be run
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH mesa 3/6] vulkan/wsi/display: pass the plane's modifiers to the image
On Thu, 27 Sep 2018 at 00:33, Keith Packard wrote: > Eric Engestrom writes: > > Signed-off-by: Eric Engestrom > > (I'll have to let someone familiar with the formats and modifiers stuff > review this one; I'm not comfortable with that at all). For my sins, I am, and this patch is: Reviewed-by: Daniel Stone I didn't have time to properly look at the others yet, but most of what you said there makes sense to me, so I'll hang on for a v2. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH 4/5] vulkan/wsi: Use the interface from the real modifiers extension
Hi,
On Mon, 1 Oct 2018 at 22:25, Jason Ekstrand wrote:
> index 70594d6c053..2850349a619 100644
> --- a/src/intel/vulkan/anv_image.c
> +++ b/src/intel/vulkan/anv_image.c
> @@ -109,6 +109,8 @@ choose_isl_tiling_flags(const struct
> anv_image_create_info *anv_info,
> case VK_IMAGE_TILING_LINEAR:
>flags = ISL_TILING_LINEAR_BIT;
>break;
> + case VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT:
> + flags = 1 << isl_mod_info->tiling;
This will break non-modifier-aware setups, as we'll call
choose_isl_tiling_flags(isl_mod_info == NULL) ...
> @@ -563,10 +562,13 @@ anv_image_create(VkDevice _device,
>
> const struct wsi_image_create_info *wsi_info =
>vk_find_struct_const(pCreateInfo->pNext, WSI_IMAGE_CREATE_INFO_MESA);
> - if (wsi_info && wsi_info->modifier_count > 0) {
> +
> + if (pCreateInfo->tiling == VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT) {
> + const VkImageDrmFormatModifierListCreateInfoEXT *mod_info =
> + vk_find_struct_const(pCreateInfo->pNext,
> IMAGE_DRM_FORMAT_MODIFIER_LIST_CREATE_INFO_EXT);
>isl_mod_info = choose_drm_format_mod(>instance->physicalDevice,
> - wsi_info->modifier_count,
> - wsi_info->modifiers);
> + mod_info->drmFormatModifierCount,
> + mod_info->pDrmFormatModifiers);
>assert(isl_mod_info);
> }
... when this branch is not taken.
Cheers,
Daniel
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH v2 0/5] Use GitLab CI to build Mesa
Hi all, On Fri, 21 Sep 2018 at 20:59, Daniel Stone wrote: > On Wed, 29 Aug 2018 at 11:13, Juan A. Suarez Romero > wrote: > > This is a first part, version 2, of a more complete proposal to use GitLab > > CI to > > build and test Mesa. This first part just adds the required pieces to build > > Mesa, using the different supported tools (meson, autotools, and scons). > > > > A second part, to be sent in the future, will use the results of the former > > to > > run different tests and verify everything works fine. > > > > An example of the pipeline that will result from this patchset can be seen > > at > > https://gitlab.freedesktop.org/jasuarez/mesa/pipelines/3070. > > > > I hope I can explain here all the rationale behind this proposal. Any > > question > > is welcomed. > > I still have an open NAK since this would DoS our existing fd.o CI > runner infrastructure; GitLab itself is fine, but subjecting our > elderly build machine with a flaky network on the wrong continent to > this would be too much. > > I'm pretty sure that I'll be able to stand up two sets of runners - > one still based in the UK, but on modern and dedicated machines with > better network; another on a US-based cloud provider, matching our > us-east GitLab master location - before XDC starts, so during XDC we > could get those as dedicated runners for a fork of the Mesa project > and try to gather some stats and profiles on how they actually > perform. > > Hopefully if it goes well then we should be able to switch them on > almost immediately. Unfortunately we're still externally stalled on this. I'd suggest that we give Juan owner access to the Mesa project in GitLab, so he can connect the existing Igalia runners to our project. These runners already do the execution for the gitlab.com project, so should be more than enough to run ours until we can get the external runners. Anyone object, agree, other? Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH v2 0/5] Use GitLab CI to build Mesa
Hi Juan, On Wed, 29 Aug 2018 at 11:13, Juan A. Suarez Romero wrote: > This is a first part, version 2, of a more complete proposal to use GitLab CI > to > build and test Mesa. This first part just adds the required pieces to build > Mesa, using the different supported tools (meson, autotools, and scons). > > A second part, to be sent in the future, will use the results of the former to > run different tests and verify everything works fine. > > An example of the pipeline that will result from this patchset can be seen at > https://gitlab.freedesktop.org/jasuarez/mesa/pipelines/3070. > > I hope I can explain here all the rationale behind this proposal. Any question > is welcomed. I still have an open NAK since this would DoS our existing fd.o CI runner infrastructure; GitLab itself is fine, but subjecting our elderly build machine with a flaky network on the wrong continent to this would be too much. I'm pretty sure that I'll be able to stand up two sets of runners - one still based in the UK, but on modern and dedicated machines with better network; another on a US-based cloud provider, matching our us-east GitLab master location - before XDC starts, so during XDC we could get those as dedicated runners for a fork of the Mesa project and try to gather some stats and profiles on how they actually perform. Hopefully if it goes well then we should be able to switch them on almost immediately. Cheers, Daniel ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH mesa 0/6] Let's get rid of 99% of warnings :)
On Fri, 21 Sep 2018 at 19:48, Jason Ekstrand wrote: > You should try building with clang. > If only there was some way we could do both, in some kind of automated fashion! ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
