Background extension has always existed in VFX — traditionally it meant painting by hand or stitching plate photography. Generative models changed the economics of the problem, but not without introducing new failure modes that took us months to work around.
What the models are actually good at
Modern inpainting models are excellent at generating photorealistic texture and plausible structure when the reference image gives them enough context. Sky replacements, extending interior walls, continuing a floor surface — these work well when the lighting in the extension zone matches the practical plate.
Where they fail
The failure modes cluster around specular detail, perspective drift, and grain mismatch. Extensions longer than about 15% of frame width start to drift from established vanishing points. Generated regions also have a different noise profile from the surrounding plate even after post-processing.
We solved grain mismatch with a frequency-domain matching pass. Perspective drift required adding a geometry constraint layer. Specular detail we still correct manually for hero shots.