From unposed photographs — one view or hundreds, no poses, no focal lengths — to a complete 3D Gaussian scene in seconds. Built on top of VGGT, and trained without any 3D ground truth.
In a recent post we looked at VGGT, the transformer that takes unposed photographs and predicts cameras, depth maps and a dense 3D point cloud in one forward pass. That point cloud can be rendered, and it gives a correct sense of the scene's 3D structure — but the renderings are sparse, not photorealistic. Photorealistic novel views require a representation such as 3D Gaussians, which have a size, an orientation and an opacity, and therefore blend into continuous surfaces. AnySplat (SIGGRAPH Asia 2025, from Shanghai AI Laboratory and collaborators) takes the same input as VGGT — uncalibrated, unposed photographs, from a single view up to hundreds — and directly predicts a complete 3D Gaussian scene, ready to render in real time, in one forward pass of a few seconds.
Let us see what AnySplat adds over existing methods. Per-scene 3D Gaussian Splatting first needs structure-from-motion to recover the cameras, which can take hours, and then adds minutes of optimization per scene on top. Feed-forward models like pixelSplat and MVSplat are fast but must be given the camera poses and work from two views. The pose-free models that followed DUSt3R, such as NoPoSplat, removed the pose requirement but, in the paper's words, are "typically limited to sparse-view inputs (e.g., 2–24 images)". AnySplat drops both requirements at once: no camera poses, and no cap on the number of views. This post explains what one forward pass produces, the architecture underneath, the training scheme — the most interesting part, because it uses no 3D ground truth anywhere — and where the model's limits are.
Given a set of photographs, AnySplat predicts three things together. First, the scene itself, as 3D Gaussians: millions of small semi-transparent colored primitives, each defined by a position, an orientation, a size, an opacity and a color, which together represent the scene and render in real time. Second, the camera parameters of every input view, intrinsics and extrinsics, expressed relative to the first camera; nothing about the cameras is given as input, everything is estimated. Third, a depth map and a confidence map for every view. One thing the model cannot know is the true size of the scene: photographs of a large room and photographs of a small model of that room look the same, and nothing in the input settles which one the camera saw. The reconstruction therefore has the right shape, but an arbitrary overall size.
The speed is what makes this practical. Thirty-two views become a scene in 1.4 seconds and sixty-four views in 4.1 seconds, on scenes where per-scene 3D Gaussian Splatting spends around 10 minutes optimizing — after structure-from-motion has already been run. A 200-view city scene reconstructs in about 33 seconds. Once predicted, the Gaussians render in real time like any 3DGS scene.
AnySplat keeps VGGT's architecture almost unchanged. Images are split into patches and encoded with DINOv2, a large pretrained vision model, then processed by a 24-layer transformer that alternates two kinds of attention: within each frame, and globally across all frames — the mechanism that lets every view see every other view. Three heads read out the result: a camera head that predicts each view's parameters, a depth head, and a new Gaussian head that predicts, for every pixel, a Gaussian's opacity, orientation, size and color, plus a confidence score. The one attribute missing from that list — the position — comes from the other two heads: each pixel is pushed out into 3D along its camera ray, to the distance given by its predicted depth, and the Gaussian is placed at that point.
AnySplat does not just imitate VGGT — it starts from it. The transformer and the depth head are initialized with VGGT's weights, and the training recipe keeps the patch embedding frozen while fine-tuning the attention layers. The whole model is about 886 million parameters, and it was trained in roughly two days on 16 A800 GPUs across nine datasets.
Before AnySplat, every feed-forward splatting model was supervised with some form of 3D ground truth: camera poses from structure-from-motion at minimum, and often depth as well. AnySplat trains on nine large datasets and uses none of their geometry annotations. The paper's claim is explicit: the model "can be trained without any 3D SfM or MVS supervision, relying solely on uncalibrated images".
The first training signal is rendering. The predicted Gaussians are rendered back into the input views through the predicted cameras, and the renderings are compared with the photographs — a pixel loss plus a perceptual one. But this signal alone is not enough, and the paper's most dramatic ablation shows why: without the second ingredient, PSNR collapses from 18.25 to 7.28. The reason is that a rendering loss on the input views has shortcut solutions. The model can paste plausible colors onto badly placed geometry: the renderings match the photographs from the input viewpoints while the 3D structure underneath is wrong. The authors describe exactly this failure — the model "tends to overfit the input views with plausible renderings without preserving accurate 3D geometry".
The second ingredient closes those shortcuts: distillation from VGGT. A frozen, pretrained VGGT processes the same images and produces pseudo-labels — its own estimated camera poses and depth maps, used as if they were ground truth. AnySplat is penalized when its predicted poses stray from the teacher's, and when its rendered depth — the depth of the Gaussians themselves, not the depth head's prediction — strays from the teacher's depth maps, with the depth comparisons restricted to high-confidence pixels. A third term ties the depth head's output to the depth of the rendered Gaussians, keeping the model's two internal representations of geometry consistent with each other. In theory, then, training needs no 3D labels at all. Keep in mind, though, what makes this possible: the supervision has not disappeared — it has moved upstream, into a teacher that was itself trained on 3D data. What AnySplat gains is practical: any collection of uncalibrated photographs becomes usable training data, with no annotation step in front of it.
Every pixel-aligned method carries the same cost: one Gaussian per pixel per view. At two views this is fine. At sixty-four views it is millions of primitives, and worse, they are redundant — overlapping photographs see the same surfaces, so the same wall gets covered with Gaussians dozens of times. The paper is direct about the consequence: per-pixel prediction works for sparse inputs but struggles "once more than 32 views are used".
AnySplat's answer is a differentiable voxelization step. Space is divided into small cubic cells — voxels — and all the Gaussians whose centers fall into the same cell are merged into a single one: each attribute of the merged Gaussian is the average of the contributing Gaussians' attributes, weighted by their predicted confidences. Because the merging is differentiable, training gradients pass through it, so the network learns which of its Gaussians to trust. Merging removes 30 to 70 percent of the primitives, and the total number of Gaussians saturates as views are added instead of growing linearly. The ablation contains a detail worth noticing: merging renders better, not worse — 3.45 million merged primitives beat the 4.82 million unmerged ones on every image-quality metric.
Evaluation is zero-shot: the test benchmarks, Mip-NeRF360 and VR-NeRF, were never seen in training, and the model is applied as is. With few input views, the comparison is against NoPoSplat and FLARE, two recent models that also work without poses. At 3 views the methods are close, and NoPoSplat is even slightly ahead in PSNR (16.36 versus 16.20). But AnySplat improves steadily as views are added, while the baselines do not, so at 16 views the gap is wide: 21.85 PSNR on Mip-NeRF360, against 15.47 for NoPoSplat and 13.21 for FLARE.
With many views, 32 to 64, the comparison is against per-scene optimization itself — 3D Gaussian Splatting and Mip-Splatting, each given about 10 minutes per scene. At 32 views, AnySplat matches them: 22.31 PSNR against 22.19, from 1.4 seconds of forward pass. At 64 views, per-scene optimization regains a small PSNR lead (21.71 versus 21.15), though AnySplat keeps the best perceptual score. And the two approaches combine: using the feed-forward output as an initialization and running roughly a thousand optimization steps — about two minutes — reaches 25.51 PSNR at 16 views, ahead of both the pure forward pass and the InstantSplat-style baseline (23.38 in three minutes). The cameras benefit as well: on RealEstate10K, unseen by both models, AnySplat's pose accuracy slightly exceeds its teacher VGGT's — the rendering loss acts as an extra multi-view consistency constraint on the cameras.
The authors list the limits plainly. The reconstruction carries no real-world scale, so applications that need true sizes — measurement, robotics — require an external reference. Reconstruction artifacts remain in the classic hard regions: skies, specular highlights, thin structures. The rendering-based training signal is less stable on dynamic scenes and under varying illumination, which is why casual video of moving subjects is not the intended input. And compute still grows with resolution and view count; the paper's stated ambition of "scaling to thousands of high-resolution inputs" is future work, not the current model.
AnySplat is the clearest example so far of a design that the strongest recent models share: start from a pretrained geometry model, attach a Gaussian head, and let distillation replace annotation. Depth Anything 3 follows the same pattern, training a Gaussian head on top of its frozen backbone, and so does Apple's SHARP, which builds on a pretrained metric-depth model. Where the earlier generations asked how to reconstruct from two posed views, this one asks only for photographs — and how the field got from there to here, stage by stage, is the subject of my overview of feed-forward 3D Gaussian Splatting.
From a single view up to hundreds. In the paper's experiments, 32 views reconstruct in 1.4 seconds, 64 views in 4.1 seconds, and a 200-view city scene in about 33 seconds — where per-scene 3D Gaussian Splatting needs around 10 minutes of optimization after structure-from-motion.
At 32 to 48 input views it matches per-scene 3DGS quality on unseen benchmarks, in seconds instead of minutes and without camera poses. At 64 views, per-scene optimization is slightly ahead in PSNR. The strongest option is the combination: AnySplat's output as initialization plus about a thousand optimization steps — roughly two minutes — beats both.
The code and the released checkpoint are both published under an MIT license — unusually permissive for this field, where DUSt3R, MASt3R and the original VGGT weights are all non-commercial. One caution: the model was initialized from VGGT weights, so check the current terms of both projects before building a commercial product on it.
AnySplat's entire training signal comes from differentiable rendering: project the predicted Gaussians into the input views, compare with the photographs, backpropagate. My 3D Gaussian Splatting Course builds exactly that pipeline from first principles in PyTorch only — no CUDA, no C++ — covering initialization, rasterization, backward passes and training loops.
Explore the Course →I share practical posts on 3DGS, NeRF, PyTorch implementations, research breakdowns, and the engineering details that usually get skipped.
Subscribe →At Qubit Analytics, we help companies put 3D Gaussian Splatting to work — classic per-scene 3DGS, feed-forward 3DGS, and the 3D foundation models covered on this page. We integrate these techniques into products, build custom reconstruction pipelines, and turn recent research into working prototypes.
For consulting inquiries:
contact@qubitanalytics.be