Depth Anything 3 predicts a single quantity per pixel — a depth and a ray — and derives the camera parameters, a dense 3D point cloud and even 3D Gaussians from it, with an unmodified DINO transformer. The result beats VGGT by wide margins. Here is how it works.
In November 2025, ByteDance released Depth Anything 3, and it did to VGGT what VGGT had done to everything before it. On camera pose estimation, DA3 nearly doubles VGGT's accuracy on several benchmarks. On geometric accuracy it leads by around 25 percent on average. And it does this with less architecture, not more: a vanilla DINO transformer, unmodified, with one prediction target. The paper reads as a thesis about minimalism, and its results are the evidence. It was accepted as an oral at ICLR 2026.
This post explains the model: what it predicts, why its output representation is the interesting part, how it is trained, and what its Gaussian-splatting head means for feed-forward 3D reconstruction. It is a companion to my VGGT explainer and to the full feed-forward 3DGS timeline, where DA3 closes the foundation-model stage.
The abstract makes two claims, and both are worth quoting because the whole paper hangs on them. First, "a single plain transformer (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization." Second, "a singular depth-ray prediction target obviates the need for complex multi-task learning."
To see what is being rejected, compare with VGGT. VGGT introduced a custom backbone design — alternating frame-wise and global attention layers — and predicts four kinds of output at once through four heads: cameras, depth maps, point maps and point tracks. DA3 keeps a pretrained DINOv2 transformer exactly as it is, and makes a single prediction per pixel: a depth and a ray. The camera parameters, a dense 3D point cloud and, with a small additional head, 3D Gaussians are all derived from that prediction. The family spans four sizes, from a 0.03B-parameter Small to a 1.1B Giant, so the claim is also tested across scale.
This is the core idea of the paper. For every pixel of every input image, DA3 predicts two things: a depth value, and a ray — six numbers describing the 3D line that the pixel looks along, as an origin point and a direction. Depth and ray together give you the 3D point directly: start at the ray's origin and walk the predicted depth along its direction. Do this for every pixel and you have a dense 3D point cloud of the scene.
The cameras come out as a by-product, and in an unusually elegant way: they are not regressed by a network but decoded from the predicted ray map using classical geometry — the same textbook algorithms used to calibrate cameras from known correspondences. The network predicts where the rays go; algebra recovers the camera that would produce them. A small auxiliary camera head exists as well, costing about 0.1 percent of the backbone's compute, but the ray map is the primary mechanism.
Why this representation rather than the pointmaps of DUSt3R and VGGT? The authors' ablation is the clearest answer in the paper: swapping the depth-plus-ray target for a depth-plus-camera target roughly halves camera pose accuracy, with the same backbone and the same data. A pointmap entangles the scene's geometry with the camera's position in a single quantity per pixel; depth and rays separate the two cleanly, and every pixel carries its own smooth, dense supervision for the camera. The representation, not the architecture, is where this paper wins.
If the backbone is unmodified, how do the views communicate? By rearranging the input rather than changing the network. In the early layers, tokens are grouped per image, so each layer's self-attention operates within one view — the standard monocular setting. In later layers, the token tensor is reshaped so that self-attention runs across all views jointly, alternating with within-view layers at a fixed ratio. The transformer never knows the difference; the same weights process whatever token arrangement they are handed. With a single input image, the cross-view arrangement simply never occurs, and DA3 naturally reduces to a monocular depth estimator — the original Depth Anything task — at no extra cost.
When camera poses are available, they can be injected: each view gets a camera token embedding its pose, and the model uses it as a hint. When they are not, a shared learnable token stands in. The outputs are read by a dual-branch dense-prediction head (DPT) — one branch for depth, one for rays, sharing their feature reassembly.
Ground-truth depth for real scenes is sparse, noisy or absent, and this is where DA3 inherits the Depth Anything lineage. A teacher model — trained in the style of Depth Anything 2, on synthetic data only — generates dense pseudo-labels for the real training images, aligned to whatever sparse ground truth exists. The student then trains on about 22 public datasets (TartanAir, ScanNet++, MegaDepth, DL3DV and others; no proprietary data), switching to teacher supervision in the later stage of training. The Giant model trained on 128 H100s for roughly ten days.
DA3 beats VGGT across the board, and the margins are not close. Averaged over the authors' new visual-geometry benchmark — camera pose, dense geometry and rendering quality across five datasets — it improves camera pose accuracy by 44.3 percent and geometric accuracy by 25.1 percent. On the hardest pose metric, the gap is starkest: on ETH3D, DA3 nearly doubles VGGT's score (48.4 versus 26.3 AUC@3), and on ScanNet++ it reaches 85.0 against 62.6. It also edges out Depth Anything 2 on monocular depth on most benchmarks, so the multi-view capability did not cost the single-image one.
One honest caveat belongs next to these numbers: the benchmark is the authors' own, assembled from existing datasets but curated by them. The margins are large and consistent enough to survive that concern.
For readers of this site, the most consequential experiment is the view-synthesis one. The authors freeze the DA3 backbone entirely and train only a small DPT head that outputs Gaussian parameters — opacity, rotation, scale and colour per pixel — on DL3DV's roughly ten thousand scenes, using eight A100s. Nothing about the geometry model is touched.
That frozen-backbone head beats the specialized feed-forward 3DGS models at their own game: 21.33 PSNR on DL3DV against 19.24 for DepthSplat, 18.13 for MVSplat and 16.55 for pixelSplat. Every one of those baselines is an architecture designed end-to-end for Gaussian prediction; DA3's head is a decoder bolted onto features that were never trained for rendering. It is the strongest evidence yet for the claim that closes my timeline article: feed-forward 3DGS has stopped being an architecture you design and become a head you attach to a geometry foundation model.
Code is Apache-2.0 on GitHub, with a Hugging Face demo. Licensing splits by size: the Base and Small weights are Apache-2.0, while the Giant and Large weights are CC BY-NC 4.0 — non-commercial, so check before shipping. A streaming variant runs long sequences in under 12 GB of VRAM, which puts serious multi-view reconstruction within reach of a consumer GPU.
VGGT's lesson was that a general transformer beats hand-designed geometric pipelines. DA3 extends the lesson one step further: even VGGT's own specialization — its custom attention pattern, its four prediction heads — was unnecessary. What mattered was choosing the right prediction target. That ordering of priorities, representation first and architecture last, is worth carrying into the next paper you read: before asking how big the model is, ask what it predicts per pixel — and what has to be entangled for that prediction to work.
DA3's Gaussian head renders through the same differentiable rasterizer as every method on this site. My 3D Gaussian Splatting Course builds that pipeline from first principles in PyTorch only — no CUDA, no C++ — covering initialization, rasterization, backward passes and training loops.
Explore the Course →I share practical posts on 3DGS, NeRF, PyTorch implementations, research breakdowns, and the engineering details that usually get skipped.
Subscribe →At Qubit Analytics, we help companies put 3D Gaussian Splatting to work — classic per-scene 3DGS, feed-forward 3DGS, and the 3D foundation models covered on this page. We integrate these techniques into products, build custom reconstruction pipelines, and turn recent research into working prototypes.
For consulting inquiries:
contact@qubitanalytics.be