NEO is a unified framework for language-guided NeRF editing in robotic manipulation. Given a prompt and a planned object motion, NEO removes the target object, reconstructs newly revealed regions with multiview-consistent progressive inpainting, and reinserts the object through direct NeRF weight editing with teacher-student knowledge distillation.
The resulting representation remains persistent across consecutive edits, enabling a robot to anticipate scene changes before execution. The paper also introduces the NEO-Dataset, a benchmark of registered tabletop scenes captured before and after object manipulation for quantitative evaluation of NeRF scene editing methods.
NEO builds an initial NeRF and language field from camera images, localizes the prompted object, then updates the scene before motion execution. The system combines object removal, multi-view inpainting, and object reconfiguration modules so the edited neural field can be reused for the next manipulation.
Progressive inpainting reconstructs surfaces that were hidden by removed objects while maintaining cross-view consistency for NeRF training.
Teacher-student distillation composes the edited and original fields so moved objects are preserved directly in the updated representation.
The NEO-Dataset contains twelve registered tabletop scenes captured before and after object removal or manipulation. Scenes are recorded with fixed robot arms and camera trajectories, then aligned with ArUco markers so rendered edited views can be compared against real post-manipulation images.
The dataset is collected in two different setups utilizing two different manipulators: the Franka Emika Panda and UR5. These provide background variability across scenes in addition to scene content variation.
Scenes are scanned in an automated manner with controlled camera motion along a fixed trajectory to ensure little-to-no variation in scene coverage. The camera poses from this process provide broad scan coverage for training and evaluating edited scene renderings.
@article{zielinski2026neo,
author={Zielinski, Mikolaj and Hall, David and Belter, Dominik and Moghadam, Peyman},
journal={IEEE Robotics and Automation Letters},
title={NEO: NeRF It Once, Edit It Many Times for Continuous Object Manipulation},
year={2026},
volume={},
number={},
pages={},
}