Neural Radiance Field (NeRF)

What is a Neural Radiance Field?

A Neural Radiance Field (NeRF) is a machine learning model that learns a scene's appearance from a collection of overlapping photographs, then renders photorealistic views from any viewpoint within or near the captured volume. Unlike traditional 3D modeling, which requires explicit geometry (meshes, surfaces, point clouds), NeRF encodes a scene as a continuous function inside a neural network. When you query that function with a camera ray direction and position, it returns the radiance (light) emitted toward the camera at that point. The result is a 3D scene representation that reconstructs fine detail and lighting but makes no guarantees about geometric accuracy or dimensional correctness.

How does NeRF work technically?

NeRF represents a scene using a multilayer neural network that takes five inputs: 3D position (x, y, z) and viewing direction (theta, phi), and outputs two values: volumetric density and radiance (color). During training, the network is fed hundreds of example rays from the input photographs, each labeled with the actual pixel color it should produce. The network adjusts its weights to minimize the difference between its predictions and the true pixel values. Once trained, the network can synthesize views from novel camera positions by casting rays through the learned scene and integrating the network's radiance predictions. The process is computationally expensive: the original Mildenhall et al. 2020 paper required 1 to 2 days of GPU time per scene. Recent improvements like Instant-NeRF reduce this to minutes.

How does NeRF differ from photogrammetry?

The distinction matters for architectural practice. Photogrammetry (structure-from-motion) converts overlapping photographs into a point cloud, mesh, or voxel volume: explicit, measurable geometry that remains constant regardless of viewing angle. A photogrammetric model can be imported into BIM software, dimensions can be verified, and the geometry is reusable across projects and clients. NeRF learns a view-dependent representation that optimizes only for visual plausibility at the captured viewpoints, unlike measured geometry or generative image approaches. Its neural function cannot directly answer "What is the distance from the window to the corner?" or "Is the facade plane or curved?" Photogrammetry answers such questions reliably; NeRF does not.

How does NeRF compare to Gaussian splatting?

Both are neural 3D reconstruction methods trained from multi-view photographs, and both avoid the labor of manual 3D modeling. The core difference is representation: NeRF uses a continuous implicit function (a neural network that you query densely), while Gaussian splatting uses an explicit set of learned 3D Gaussians that rasterize directly to 2D. This makes Gaussian splatting much faster to render (often real-time at 60 FPS in a VR viewer), but it can exhibit popping and temporal artifacts when the camera moves through or near a splat. NeRF is slower to render (often 5 to 30 seconds per frame), but produces smoother, more photorealistic single images. For architectural visualization, choose NeRF if you want the best-looking still render for client presentations or heritage documentation; choose Gaussian splatting if you need interactive real-time walkthroughs or VR headset performance.

What are the limitations and honest use cases?

NeRF excels at rendering from novel viewpoints but does not produce a measured survey. A client seeing a NeRF walkthrough around their building extension should understand it as a visualization of appearance, not a proof of dimensional compliance. NeRF cannot be used to verify that a window fits the opening, that a staircase meets code, or that a facade meets zoning setbacks. It is also training-data-hungry: poor lighting, repetitive patterns, or thin features (like railings) often cause artifacts. The capture process requires many overlapping photos from a careful path around the subject; handheld photography works, but drone or structured paths produce better results. For a 50-meter facade with complex geometry, you may need 100 to 200 images and several hours of training.

What tools and workflows should architects know?

Consumer and professional NeRF tools are rapidly maturing. Web-based services (Luma AI, Cinematic 360, RealityScan) accept photo uploads and output interactive 3D models without requiring GPU setup; processing takes hours to days. Open-source frameworks (Instant-NeRF, NeRF-PyTorch) run on local or cloud GPUs and offer more control over training parameters. Some photogrammetry platforms (Metashape, Pix4D) are adding neural rendering as an alternative output mode. For architects, the practical workflow is: shoot 50 to 100 overlapping photos with a smartphone or DSLR in good daylight; upload to a cloud NeRF service; in 24 to 48 hours, receive a navigable 3D model that you can embed in a presentation, VR platform, or client website. A walkthrough video can be generated automatically from the NeRF model. The cost is typically 5 to 50 euros per scene.

Integration with design workflow is emerging. Some architectural practices photograph existing sites, train a NeRF, then compose the client's design into the NeRF context using a 3D editor or blending tool. This delivers photorealistic site integration faster than manually texturing a photogrammetric mesh or rendering a design against a backdrop image.