Give the network two photographs — no poses, no focal lengths — and it returns a dense 3D point cloud, one point per pixel. The camera parameters are then estimated from that point cloud. MASt3R builds on top of DUSt3R and makes the result precise and metric.
In a recent post we looked at pixelSplat and MVSplat: feed-forward models that turn two photographs into a 3D Gaussian scene in a fraction of a second. Both share one requirement — the camera parameters of the input images must be known: positions, orientations, focal lengths. In practice they come from structure-from-motion — COLMAP, in most pipelines — which matches the images and then repeatedly runs an expensive optimization called bundle adjustment. That process takes minutes to hours, needs many overlapping images, and fails outright on scenes with little texture or few views. The "single forward pass" only begins after it has succeeded.
DUSt3R (CVPR 2024, NAVER LABS Europe) removed that requirement. It takes two photographs with no camera information at all and directly regresses the 3D structure they show; the cameras are then estimated from that structure, by small optimizations that take seconds. Camera parameters stopped being an input to reconstruction; they are read off its result. MASt3R (ECCV 2024, an oral, from the same lab) kept the architecture and fixed its two weaknesses, adding precise image matching and metric scale. Together they are the foundation under nearly every pose-free method in feed-forward 3D Gaussian Splatting. This post explains how they work, what they measurably changed, and where their limits are.
Everything in DUSt3R follows from its output representation. For each input image, the network predicts a pointmap: for every pixel, the 3D coordinates of the surface point that pixel observes. An image of width W and height H becomes a W×H×3 array — a dense point cloud with one point per pixel, in one-to-one correspondence with the image.
The decisive design choice is the coordinate frame. Both pointmaps — the first image's and the second's — are expressed in the coordinate frame of the first camera. The network therefore does not just estimate each view's geometry; it places both views' geometry in a single shared 3D space. Where the two images see the same surface, their pointmaps must produce the same 3D points. The relative pose between the cameras is never predicted anywhere, yet it is implicitly contained in the output, because view two's geometry is expressed in view one's frame.
Alongside each pointmap, the network predicts a per-pixel confidence. Some pixels have no well-defined 3D point — the sky, or translucent surfaces — and the confidence lets the model say so instead of being forced to guess. Notice also what the network does not contain: no camera model. Classical reconstruction bakes the pinhole projection equations into every step; DUSt3R's authors describe their formulation as relaxing the hard constraints of the usual projective camera models. The network just maps pixels to 3D points, and geometry is imposed only later, when cameras are extracted. Depth, for instance, is nothing special here — it is simply the z-coordinate of each point. Even the single-image case needs no separate treatment: feed the same photograph in twice, and the pairwise model becomes a monocular one.
The architecture is deliberately plain. A ViT-Large encoder processes both images with shared weights. Two ViT-Base decoders, one per view, then exchange information at every block through cross-attention: each view's tokens attend to the other view's tokens, so each image is interpreted in the context of the other — which is what makes the shared coordinate frame learnable at all. A DPT head on each branch regresses the pointmap and its confidence. The network is initialized from CroCo pretraining, a self-supervised task in which a heavily masked image must be reconstructed with the help of a second view of the same scene — practice, in effect, for transferring geometry between viewpoints. The published model is trained on a mixture of eight datasets — indoor, outdoor, synthetic, driving — totalling 8.5 million image pairs, first at 224×224 resolution and then at 512 pixels.
The loss is a regression in 3D: the distance between each predicted point and its ground-truth counterpart. Two details matter. First, scale. The training scenes come with arbitrary scale — the same ambiguity pixelSplat had to engineer around — so prediction and ground truth are each normalized by their own average point distance before comparison. The normalizer is computed over both views of the pair jointly, not per view: normalizing each view separately would destroy the relative scale between the two views, which is exactly the information the shared frame is supposed to carry. Second, confidence. Each pixel's regression error is weighted by the predicted confidence, minus a penalty on low confidence, so the model can down-weight pixels whose geometry is unknowable but pays a price for doing so — it cannot simply declare everything uncertain.
Nothing about the cameras is predicted, and everything about them is recoverable. The focal length follows from the first view's pointmap alone: assuming the principal point is near the image centre, the focal length that best projects the predicted 3D points back onto their own pixels is found by a fast iterative solver, in a few iterations. The relative pose between the cameras follows from comparing the two predictions of the shared geometry — either by directly aligning point clouds (Procrustes alignment), or, more robustly, by solving a standard pose problem with RANSAC on correspondences. Correspondences themselves come almost for free: two pixels match if their predicted 3D points are nearest neighbours in the shared space. One network output, and matching, depth estimation, focal estimation and pose estimation all reduce to small computations on top of it.
The network only ever sees two images at a time. Reconstructing a collection works by running it on every useful pair — pairs of images that share visual content, organized as a graph — and then merging all the pairwise predictions. The merging step optimizes, for each image, a pointmap in a single world frame, together with one rigid transformation and one scale factor per pair, so that each pairwise prediction agrees with the world-frame geometry.
It is tempting to call this bundle adjustment, and the paper is explicit that it is not. Bundle adjustment minimizes 2D reprojection errors through the camera model; DUSt3R's global alignment minimizes 3D distances between pointmaps directly. That makes it a much easier optimization problem: plain gradient descent converges in a few hundred steps, taking seconds on a GPU where structure-from-motion takes minutes to hours. The caveat is that this is still an optimization, not a forward pass — and the number of pairs grows quadratically with the collection. Both points define the follow-up research: NAVER's own MUSt3R and Meta's Fast3R rebuilt the network to ingest many views at once, and VGGT eventually removed the post-optimization entirely.
The results were remarkable for a single set of weights evaluated across tasks it was never tuned for. On multi-view pose estimation, the field's standing benchmark numbers moved by a margin rarely seen in one paper: on CO3D, the fraction of image pairs with rotation error under 15 degrees rose from 80.5% for the best prior learned method, PoseDiffusion, to 96.2%. As a monocular depth estimator — again, the same weights, with the image simply fed in twice — it comes close behind supervised specialists trained for that single task (an absolute relative error of 6.50 versus 5.40 for DPT on NYUv2). On the DTU multi-view stereo benchmark it reaches 1.741 mm of overall error zero-shot, without ever training on DTU.
That last number also marks the boundary. Classical pipelines that triangulate with known cameras reach 0.578 mm on DTU, three times better, and the paper says plainly why: regression is less accurate than sub-pixel triangulation. Two further limits matter. The reconstruction is only defined up to an unknown global scale — DUSt3R cannot tell a large room from a small model of one. And while nearest-neighbour matching in pointmap space is strikingly robust to extreme viewpoint changes, it is not precise: the pointmaps are smooth regressed fields, and matches inherit their blur. Precision and scale are exactly what MASt3R adds.
MASt3R starts from an observation about image matching, the task of finding which pixel in one image corresponds to which pixel in another — the raw material of localization and mapping systems. Matching had always been treated as a 2D problem, yet it is fundamentally a 3D one: two pixels correspond because they observe the same point of a 3D scene. So instead of building a better 2D matcher, MASt3R grafts matching onto DUSt3R's 3D reconstruction — the paper's title, "Grounding Image Matching in 3D," is a literal description.
Concretely, MASt3R keeps the entire DUSt3R architecture and adds a second head that outputs a dense local feature map: a 24-dimensional descriptor per pixel, trained with a contrastive loss (InfoNCE) so that descriptors of truly corresponding pixels — known from the ground-truth geometry — match each other and nothing else. The pointmaps provide robustness, deciding roughly where in the scene a pixel belongs; the descriptors provide the precision the pointmaps lack.
The second change is a small modification of the loss that buys metric scale. DUSt3R normalizes the prediction by its own scale, so the network is never asked to know true size. MASt3R, whenever a training scene has metric ground truth, normalizes the prediction by the ground truth's scale instead — and the only way to make that loss small is to predict distances at their true metric values. The released model predicts metric-scale pointmaps.
The third change makes dense matching affordable. Finding all mutual nearest-neighbour matches between two feature maps naively compares every pixel with every pixel — quadratic in image size, prohibitively slow at high resolution. MASt3R's fast reciprocal matching starts from a sparse grid of seed pixels and repeatedly maps each one to its best match in the other image, then maps that match back — collecting, along the way, the pairs that map onto each other, which are kept as correspondences. This drops the complexity to linear in image size, and the paper reports a finding the authors themselves flag as surprising: subsampled matching is not just faster but more accurate, because points that survive the iterated reciprocal test are disproportionately correct matches — the procedure filters outliers as a side effect.
The clearest test is a benchmark called map-free relocalization. It works like this: you are given a single photograph of a place. Someone then takes a new photograph of the same place, and the task is to estimate where that second camera was — its position in metres and its viewing direction — from the two images alone. Localization systems normally rely on a 3D map built beforehand from many photographs of the scene; here there is no map, and a single pair of images contains nothing that reveals the scene's true size. The task therefore demands exactly the two things MASt3R added: matching that survives large viewpoint changes, and metric-scale geometry. The numbers reflect it: the median pose error drops from roughly a metre with DUSt3R to 0.36 m and 2.2°, and on the benchmark's headline score — an area-under-curve measure between 0 and 1 — MASt3R reaches 0.933, against 0.697 for DUSt3R and 0.634 for the best method published before them.
The DTU boundary moves too. Triangulating MASt3R's matches — precise enough, now, to triangulate — cuts DUSt3R's 1.741 mm to 0.374 mm, within reach of the best DTU-trained specialist (0.295 mm) while still never training on DTU. And on multi-view pose estimation on RealEstate10K, MASt3R adds almost nine points on top of DUSt3R's already large improvement. The pattern across benchmarks is consistent: DUSt3R made pose-free reconstruction work; MASt3R made it accurate.
Neither paper predicts a single Gaussian, and yet they sit at the root of the pose-free branch of feed-forward 3DGS. Splatt3R made the connection first and most literally: freeze MASt3R entirely, train a small head that turns each predicted 3D point into a Gaussian, and pose-free splatting from an uncalibrated pair follows. NoPoSplat took the deeper lesson — predict all geometry directly in the first camera's frame — and designed a Gaussian prediction network around it, trained end to end rather than bolted onto a frozen backbone, with results strong enough to beat baselines that were given the ground-truth poses; that story continues in my overview of feed-forward 3D Gaussian Splatting, in the stage where camera poses stop being required. Around the pair of papers, an ecosystem formed: MASt3R-SfM and MASt3R-SLAM rebuilt structure-from-motion and real-time SLAM on top of the model, and MUSt3R and Fast3R removed the two-view bottleneck.
The lineage also shows where the design reached its limit. VGGT, CVPR 2025's Best Paper, positions itself against exactly the two properties flagged above: in its authors' words, DUSt3R and MASt3R "can only process two images at once and rely on post-processing to reconstruct more images." VGGT processes hundreds of views in one forward pass with no global alignment afterwards, and outperforms both. That does not diminish what the pair of papers did — it confirms it. The representation VGGT regresses is the pointmap, the convention it uses is the first-camera frame, and the question it answers — can cameras be an output rather than an input? — was settled by DUSt3R.
MASt3R is DUSt3R with three additions: a second head that outputs dense local features for precise image matching, a modified loss that makes the predicted geometry metric-scale rather than scale-ambiguous, and a fast reciprocal matching scheme that makes dense matching affordable at high resolution. The backbone, the pointmap output and the shared-frame convention are unchanged. In practice the additions make MASt3R the stronger model on reconstruction benchmarks as well, so it is the one to build on.
Yes, but not in one forward pass. The network processes pairs; a larger collection is handled by running it on all overlapping pairs and merging the predictions with a global alignment optimization carried out directly in 3D — not a bundle adjustment. It converges in a few hundred gradient steps and takes seconds on a GPU, though the pairwise cost grows quadratically with the number of images.
Not under the official releases. Both repositories are distributed under a CC BY-NC-SA 4.0 license — non-commercial use only — and the trained checkpoints carry the same restriction, with additional notices inherited from the training datasets' licenses.
The methods that turn DUSt3R's point clouds into 3D Gaussians all train the same way: render the Gaussians with a differentiable rasterizer, compare against real photographs, backpropagate. My 3D Gaussian Splatting Course builds that rasterizer from first principles in PyTorch only — no CUDA, no C++ — covering initialization, rasterization, backward passes and training loops.
Explore the Course →I share practical posts on 3DGS, NeRF, PyTorch implementations, research breakdowns, and the engineering details that usually get skipped.
Subscribe →At Qubit Analytics, we help companies put 3D Gaussian Splatting to work — classic per-scene 3DGS, feed-forward 3DGS, and the 3D foundation models covered on this page. We integrate these techniques into products, build custom reconstruction pipelines, and turn recent research into working prototypes.
For consulting inquiries:
contact@qubitanalytics.be