VGGT takes up to hundreds of images and predicts the camera poses, the depth maps and a dense 3D point cloud in a single forward pass — no structure-from-motion, no optimization. Here is how it works, and why it won Best Paper at CVPR 2025.
In March 2025, a single model made most of the specialized 3D reconstruction pipeline unnecessary. VGGT (Visual Geometry Grounded Transformer), from Oxford's Visual Geometry Group and Meta AI, takes anywhere from one to hundreds of images and predicts, in a single forward pass, everything the classical pipeline used to produce: the camera parameters of every image, a depth map for every image, and a dense 3D point cloud of the scene. No structure-from-motion beforehand, no optimization afterwards. It won Best Paper at CVPR 2025, and within months it had become the backbone that newer reconstruction methods build on rather than compete with.
This post explains how it works, what it actually predicts, what the numbers say, and what it means for feed-forward 3D reconstruction. It is a companion to my longer overview, Feed-Forward 3D Gaussian Splatting: From Proof of Concept to Production, where VGGT opens the foundation-model stage of the story.
Before VGGT, recovering geometry from a set of photographs meant chaining specialized components. The classical route runs structure-from-motion (COLMAP) to estimate cameras, then multi-view stereo for dense geometry — minutes of processing, and failure on scenes with little texture. The 2024 alternative, DUSt3R, replaced much of that chain with a network that takes two images and directly predicts a 3D point for every pixel of both. That representation is sufficient on its own: once you know where every pixel lives in 3D, the depth maps follow immediately, and even the camera poses can be recovered from the predicted points. But the network only handles two images at a time, so larger collections require stitching the pairwise predictions together with a global alignment optimization. Either way, an optimization step sits somewhere in the pipeline.
VGGT's claim is that none of this structure is necessary: one transformer, trained on enough data, predicts the cameras, the depth maps and the 3D point cloud at once, for any number of input images — and does it better than the pipelines it replaces.
Each input image is tokenized by a pretrained DINOv2 encoder. To each image's tokens, VGGT appends one camera token and four register tokens — extra learnable tokens that will later carry that image's camera information. The resulting sequence, all images together, passes through 24 transformer blocks totalling about 1.2 billion parameters.
The one architectural novelty is how attention is organized. The blocks alternate between frame-wise self-attention, where each image's tokens attend only to tokens of the same image, and global self-attention, where every token attends to every token across all images. The frame-wise layers build a coherent understanding of each image on its own; the global layers exchange information between viewpoints — implicitly doing the job that epipolar constraints, cost volumes and pairwise matching did in earlier systems. There is no cross-attention and no geometric machinery anywhere in the network: it is self-attention all the way through.
The transformer itself only produces features; four small output heads turn those features into predictions. The camera head reads each image's camera token and predicts that image's camera: its orientation as a quaternion, its position, and its field of view — nine numbers in total, with the principal point assumed to be at the image centre. The depth head predicts a depth value for every pixel, and the point head predicts a 3D point for every pixel, which together form a dense point cloud of the scene; both are standard dense-prediction decoders (DPT). Finally, the tracking head, built on the CoTracker2 architecture, takes any pixel you select in one frame and predicts where that same physical point appears in every other frame.
Every prediction needs a coordinate system to be expressed in, and VGGT uses the same convention as DUSt3R: the first camera is the coordinate system. Its pose is not predicted — it is defined to sit at the origin, and everything else is expressed relative to it: the poses of the other cameras, the depth maps, the whole point cloud. The network therefore has to know which of its inputs is the first image, and the mechanism for that is remarkably simple: the first image is given its own set of learnable tokens, different from the set shared by all the other images. Nothing else distinguishes it. One practical consequence: the order of the remaining images does not matter — shuffle them, and the predictions are identical, just reordered.
VGGT is trained on seventeen datasets spanning indoor scans, outdoor scenes, synthetic renders, driving footage and object-centric captures — among them Co3Dv2, ScanNet, MegaDepth, DL3DV, Kubric, Hypersim and Virtual KITTI. The loss is a weighted sum over the four tasks: a Huber loss on the cameras; depth and point losses weighted by the model's own predicted uncertainty; and a tracking term. Training took 64 A100 GPUs for nine days at a maximum image dimension of 518 pixels. Large by academic standards; small by the standards of language models — and this scale of data, not any architectural insight, is what the specialized pipelines could not match.
VGGT beats DUSt3R and MASt3R on every task the three share, usually by a wide margin. On camera pose estimation it scores 88.2 AUC@30 on Co3Dv2, against 81.8 for MASt3R and 76.7 for DUSt3R — and it produces that answer in 0.2 seconds, where the other two need an optimization stage on top of their forward pass. For depth estimation on DTU, it reaches the accuracy that MASt3R only achieves when given the ground-truth cameras, while receiving no camera information at all. Its point clouds are the most accurate of the three on ETH3D. And its features improve point tracking when plugged into an existing tracker.
Two results deserve more attention than they usually get. First, the forward pass and optimization are complementary. On its own, VGGT already outperforms the methods that rely on an optimization stage — DUSt3R and MASt3R with their global alignment, and VGGSfM, a learned structure-from-motion pipeline that uses bundle adjustment. But you can also run bundle adjustment on top of VGGT, using its predictions as the starting point, and it still helps: camera accuracy on Co3Dv2 rises from 88.2 to 91.8. The forward pass does not make optimization useless; it replaces the part that was expensive — starting from nothing.
Second, the dedicated point head turns out to be redundant. VGGT offers two ways to obtain the 3D point cloud: take the point head's output directly, or combine two of its other outputs — use the predicted camera parameters to lift each pixel's predicted depth into 3D. The second route is more accurate than the head that was built for the job (0.677 versus 0.709 Chamfer distance on ETH3D). Predicting depth and cameras separately and composing them beats predicting their combination directly.
Speed scales gracefully with the number of views. On an H100 with FlashAttention, one frame takes 0.04 seconds and 1.9 GB of memory; fifty frames take about a second and 11 GB; two hundred frames take under nine seconds and 41 GB.
VGGT predicts geometry, not appearance. Its outputs contain no colour and no Gaussians, so there is no scene you can render from a new viewpoint. The paper does include a novel-view-synthesis experiment, but it takes a shortcut: the network is given the target viewpoint and generates that image directly, pixel by pixel, without ever building a 3D scene in between. It gets close to a specialized view-synthesis model this way, while training on roughly a fifth of the data. Rendering a scene you can move through freely still requires attaching a Gaussian head, which is exactly what AnySplat did ten weeks after release — using VGGT itself as the teacher that supplies geometry supervision.
The authors state the current limits plainly: no support for fisheye or panoramic images, degraded results under extreme camera rotations, and tolerance for only minor non-rigid motion — a scene with substantial deformation breaks the model.
Code and weights are on GitHub. Licensing needs one minute of attention: the original VGGT-1B checkpoint is non-commercial, but since July 2025 the code is commercially usable and a separate VGGT-1B-Commercial checkpoint is available on request — it scores marginally better than the original. A memory optimization released in May 2026 roughly doubled the number of frames that fit on a GPU, and the same group has since released VGGT-Omega, a scaled successor extending the approach to dynamic scenes. If you build on this line of work, track the repository rather than the paper.
For two years, feed-forward 3D reconstruction meant designing geometric structure into networks: epipolar attention in pixelSplat, cost volumes in MVSplat, pointmap regression in DUSt3R. VGGT demonstrated that a plain transformer at sufficient scale learns whatever structure it needs — and generalizes better for it. The methods that came after treat reconstruction as a fine-tuning problem on top of a geometry backbone rather than an architecture design problem, and the strongest 2025 results in feed-forward 3DGS are built either on VGGT directly or on the paradigm it validated. That shift, from designing pipelines to attaching heads, is the subject of the second half of the full timeline.
The Gaussian heads that get attached to backbones like VGGT all render through the same differentiable rasterizer. My 3D Gaussian Splatting Course builds that pipeline from first principles in PyTorch only — no CUDA, no C++ — covering initialization, rasterization, backward passes and training loops.
Explore the Course →I share practical posts on 3DGS, NeRF, PyTorch implementations, research breakdowns, and the engineering details that usually get skipped.
Subscribe →At Qubit Analytics, we help companies put 3D Gaussian Splatting to work — classic per-scene 3DGS, feed-forward 3DGS, and the 3D foundation models covered on this page. We integrate these techniques into products, build custom reconstruction pipelines, and turn recent research into working prototypes.
For consulting inquiries:
contact@qubitanalytics.be