Two papers, one benchmark, opposite philosophies: predict depth from features, or measure it by matching across views. MVSplat won the head-to-head — and larger models later overtook both.
In the winter of 2023–2024, two papers attacked the same problem with opposite philosophies. pixelSplat bet that a network should predict depth from image features, treating it as a learning problem. MVSplat, four months later, bet that depth should be measured by matching features across views, treating it as a geometry problem. In plainer terms: one asks a neural network to guess how far away everything is, the other checks, distance by distance, where the two photographs agree. Both were evaluated on the same benchmark under the same conditions, which makes this one of the cleanest controlled comparisons in recent 3D vision — and the outcome shaped everything that followed in feed-forward 3DGS. This post explains both methods, the numbers that settled the argument, and the part of the story that only became visible a year later.
Both papers work under the same protocol. The dataset is RealEstate10K: 67,477 indoor video sequences from YouTube property tours, with camera poses for every frame computed beforehand by structure-from-motion. The model receives two input frames from a video sequence, at 256×256, and predicts a set of 3D Gaussians. Evaluation uses held-out frames of the same sequence: the predicted scene is rendered from the held-out frames' camera viewpoints, and the rendered images are compared with the real ones. Both methods predict Gaussians pixel-aligned to the inputs, and both require the camera poses to be given. Everything else about them differs.
pixelSplat (CVPR 2024, Best Paper Honorable Mention) identified the two obstacles that define this task. The first is scale: structure-from-motion reconstructs each training scene only up to an arbitrary scale factor, so a model trained across thousands of scenes cannot learn a consistent notion of distance — it has to read the scale off the camera pair it is given. pixelSplat's mechanism for this is an epipolar transformer: for a pixel in one image, the matching content in the other image must lie on a known line (the epipolar line), so features attend along that line, and each sample on it is annotated with its triangulated depth, which carries the scene's scale into the features.
The second obstacle is that gradients through 3D Gaussians are local: a primitive placed in the wrong spot receives almost no useful gradient, because the rendering loss barely depends on it once it sits far from where it should be. pixelSplat's answer is to avoid predicting positions directly. It predicts a probability distribution over depth along each ray, samples three depths per ray, places a Gaussian at each, and sets each Gaussian's opacity to the predicted probability of the depth it was sampled at. Geometry then improves by reshaping a probability distribution rather than by dragging misplaced primitives through space — this reparameterization is what makes the model trainable.
The cost is size and geometry. The model has 125.4 million parameters and takes about 0.1 seconds per pair. And while its renderings score well, the 3D structure beneath them is weak: viewed from far outside the input viewpoints, a pixelSplat reconstruction is a cloud of floating Gaussians, and the authors add 50,000 extra training steps with a depth regularizer to make the geometry presentable.
MVSplat (ECCV 2024) replaces the learned depth branch with a classical idea made differentiable: instead of asking a network what a pixel's depth is, test every plausible depth and see which one makes the two views agree. Concretely, the model picks 128 candidate depths. For each candidate, it asks: if this pixel really were at that depth, where would it appear in the other view? Using the known cameras, it looks up the other view's features at that location and compares them with the pixel's own features. If they match, the candidate depth is probably right — both cameras are looking at the same surface. This gives every pixel a score for each of the 128 candidates; the collection of scores is the plane-sweep cost volume. A small U-Net cleans up the scores, a softmax turns them into probabilities, and the pixel's depth is the probability-weighted average of the candidates. Finally, each pixel is placed in 3D at its estimated depth along the camera ray, and that 3D point becomes the centre of the pixel's Gaussian.
One detail mirrors pixelSplat beautifully: opacity comes from the maximum of that softmax. Confidence that a surface was matched becomes confidence that a Gaussian should be visible. Both papers, in other words, route their uncertainty into opacity — pixelSplat from a predicted distribution, MVSplat from measured agreement between views. It is the same insight arrived at from opposite directions.
On RealEstate10K, MVSplat wins on every axis at once: 26.39 versus 25.89 PSNR, with 12 million parameters against 125.4 million, in 0.044 seconds against 0.104 — and trained with a photometric loss alone, no depth-regularized finetuning stage, yet with visibly cleaner geometry. Cross-dataset transfer widens the gap: trained on RealEstate10K and tested zero-shot, MVSplat leads on ACID and by a larger margin on DTU, where the domain gap is largest — its zero-shot ACID score even slightly surpasses the pixelSplat model that was trained on ACID.
Two experiments in the MVSplat paper turn a scoreboard into an argument. First, the ablation: remove the cost volume from MVSplat and PSNR collapses from 26.12 to 22.83 — the single most important component, by far. Second, and more convincing, the transplant: the authors install their cost volume inside pixelSplat's own architecture, replacing its probabilistic depth branch, and pixelSplat improves from 25.89 to 26.63. The mechanism works in the rival's body. That is about as close to a controlled experiment as method comparisons get, and it is why the field read the result as "matching beats regression" rather than "MVSplat beats pixelSplat."
The comparison has a second act, and it matters if you are choosing an approach today. The cost volume won at 2024's data scale: a geometric prior is a way of not having to learn geometry from a single dataset. But VGGT then showed that a plain transformer trained on seventeen datasets learns the geometry without any matching machinery, and Depth Anything 3 beat VGGT with even less structure. NoPoSplat, meanwhile, outperformed both pixelSplat and MVSplat on their own benchmark without being given the camera poses both of them require. The honest summary: matching beat regression head-to-head, and then scale beat both — which is the arc of the entire field, told through two papers.
What survives from each is a technique, not a scoreboard position. From pixelSplat: opacity as sampled probability, and the sharpest statement of the scale-ambiguity problem. From MVSplat: the demonstration that a differentiable classical prior can replace a hundred million learned parameters — and its cost volume lives on inside DepthSplat and VolSplat, the strongest reconstruction models of 2025.
On the benchmarks both papers use, yes, and it is not close: higher quality, ten times fewer parameters, twice the speed, better geometry and better cross-dataset transfer. pixelSplat remains the paper to read for the clearest treatment of the scale-ambiguity problem, and it defined the benchmark protocol everyone since has used.
A structure that tests many candidate depths for every pixel. For each candidate depth, the second view's features are warped into the first view as if that depth were correct, and compared with what is actually there; agreement is evidence that a surface sits at that depth. Sweeping through all candidates gives, per pixel, a score for every depth — from which the best depth is read off with a softmax.
Yes, both take the two cameras' poses and intrinsics as input — in practice from structure-from-motion. Removing that requirement is the next chapter of the field: NoPoSplat later matched and beat both models without pose inputs, and the story from there is covered in the timeline article.
Both methods render through the same differentiable rasterizer. My 3D Gaussian Splatting Course builds that pipeline from first principles in PyTorch only — no CUDA, no C++ — covering initialization, rasterization, backward passes and training loops.
Explore the Course →I share practical posts on 3DGS, NeRF, PyTorch implementations, research breakdowns, and the engineering details that usually get skipped.
Subscribe →At Qubit Analytics, we help companies put 3D Gaussian Splatting to work — classic per-scene 3DGS, feed-forward 3DGS, and the 3D foundation models covered on this page. We integrate these techniques into products, build custom reconstruction pipelines, and turn recent research into working prototypes.
For consulting inquiries:
contact@qubitanalytics.be