GeoStyler: A Generalizable Geometry-aware Diffusion-based Approach for Direct 3D Gaussian Style Transfer

Direct 3D scene stylization from sparse views remains a significant challenge, as existing optimization-based methods are prohibitively slow and require dense inputs to prevent geometric corruption. While recent direct methods accelerate this process, their rigid decoupling of a static geometry from appearance often leads to visual artifacts, where stylistic textures conflict with and distort the underlying scene structure. To address these limitations, we introduce GeoStyler, a direct framework that generates high-fidelity, multi-view consistent stylized 3D scenes in seconds. Our approach reformulates the conventional pipeline by first leveraging a diffusion model to generate a set of geometrically consistent stylized 2D images. The core of this stage is a novel hybrid query formulation for the self-attention mechanism. Specifically, cross-view geometric information is directly embedded into the query to enforce 3D consistency, while style information is independently injected via the key and value to preserve scene structure. This process is further stabilized by a geometrically-aware latent initialization that provides a coherent starting point for the denoising process. Subsequently, a decoupled reconstruction network lifts these 2D stylized images to 3D Gaussians. A geometry branch predicts a robust 3D scaffold from the original content images, while a parallel style branch predicts the final appearance from our generated stylized images, ensuring structural integrity is not compromised. Extensive experiments on large-scale benchmarks, including RealEstate10K and ACID, demonstrate that GeoStyler significantly outperforms prior arts in stylization quality and multi-view consistency, achieving state-of-the-art performance with a dramatic speedup.

Overview of GeoStyler. GeoStyler employs a decoupled, dual-branch architecture. The style branch generates multi-view consistent stylized images via a diffusion-based process, while the parallel geometry branch concatenates the per-view cost volumes and monocular features with a 2D U-Net, which can predict the geometric attributes of the scene, including opacity, 3D covariance matrix, and depth maps from the original images. The final stylized 3D Gaussians are synthesized by combining the color from the stylized images with these geometric attributes, using the unprojected depth maps to establish the Gaussian means.

Detailed structure of the style branch. The core of the method lies in the reverse process, where we decouple information within the attention mechanism: style is injected via the key and value from the style image. At the same time, multi-view consistency is enforced through a shared hybrid query built using cross-view attention. Our hybrid initial latent AdaIN bookends this denoising process to ensure a consistent starting point, and a final step using geometric constraint to refine the output.