Issue: When we run picktest.DragCube3D with Metal API validation enabled and resize the frame(or move the red cube) we get assertion error for depthAttachmentPixelFormat.
Root cause: When are doing both 2D and 3D rendering on same RTT, we need to use render pipeline state with correct depthAttachmentPixelFormat on the render encoder, even when depth testing is no enabled. We are getting this assertion in drawIndexedQuads because in this code flow depth testing is disabled and RTT has depth attachment. Since we continue to use the encoder which is created with render pass descriptor having a depth texture, it expects us to set render pipeline state with MTLPixelFormatDepth32Float. Also we don't see any artifacts because depth testing is disabled and we are just not setting proper pixel format for this unused depth texture. Fix: Pick render pipeline state for encoder based on whether encoder is created with depth attachment or not instead of depth testing state. Changes are made to make sure we are updating depth texture related information and set proper pipeline state. Also setting depthStencilState should be unconditional so that we flip properly when depth testing state changes on the same encoder. Functional testing is done using Ensemble8, FX83DFextures with Metal validation and multi monitor setup, i don't see any issues. Performance testing is done using RenderPerfTest and there are no regressions. We don't have regression test because this assertion error happens only when Metal API validation is enabled and also the issue doesn't result in any corruption. --------- - [x] I confirm that I make this contribution in accordance with the [OpenJDK Interim AI Policy](https://openjdk.org/legal/ai). ------------- Commit messages: - 8378271: DragCube3D fails with Metal validation assertion when resized Changes: https://git.openjdk.org/jfx/pull/2318/files Webrev: https://webrevs.openjdk.org/?repo=jfx&pr=2318&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8378271 Stats: 48 lines in 4 files changed: 15 ins; 8 del; 25 mod Patch: https://git.openjdk.org/jfx/pull/2318.diff Fetch: git fetch https://git.openjdk.org/jfx.git pull/2318/head:pull/2318 PR: https://git.openjdk.org/jfx/pull/2318
