Physically Based Rendering (PBR) is the physical simulation of light transport within a scene, i.e., how light participates in events such as reflections, refractions and scattering when interacting with objects. Recent advances have made the diffentiation of PBR possible and computationally tractable, allowing the reconstruction of physical scene properties (such as lighting, materials, shapes, etc.) from two-dimensional images using gradient-based optimisation. This enables us to optimise physically-interpretable parameters instead of reasoning about pixel-based images. Here we demonstrate the potential of differentiable PBR in two settings: (a) to find physically based model metamers to explain model sensitivities and invariances to physical object properties and (b) to create controversial scenes where two models disagree on the interpretation of a physical scene, using gloss appearance as an example. These examples display the promise of differentiable PBR for understanding perceptual inferences in humans and machines.

I will give a remote talk about our latest preprint at Vision Science Society 2026 about our latest preprint MRD.