Drivable generalized head neural radiance field

Subjects: Computer Science >> Other Disciplines of Computer Science submitted time 2023-03-01

Wang Yue

Abstract： In recent years, with the rapid development of computer vision, the concept of digital human has attracted wide attention from all walks of life, and the modeling of high-fidelity human body, head and hand has been deeply studied. This paper focus on head modeling and propose a generalized head model based on neural radiance field, which is parameterized by face recognition network and 3D face morphable model, therefore, it can directly control the semantic attributes such as identity and expression of the generated image, and support freely modifying the rendering pose of the image. In order to improve the rendering speed of neural radiance field, this paper use the combination of the volume rendering and two-dimensional neural rendering to replace the traditional pure volume rendering, which can speed up the rendering process while preserving image quality. The head model can render images with the speed of 15 frames per second on the Tesla V100 GPU. By collecting a large amount of head RGB images data to participate in training stage, the model can generate high-fidelity rendering images, and also have realistic fitting results on the test set, it can be generalized to new identity and expression that have not been trained. Thanks to the ability of implicit representation of 3D geometric scene by neural radiance field, the rendering results of the model has multi-view consistency, and has many uses such as novel view synthesis, expression transfer, driving and so on.

Peer Review Status:Awaiting Review

 Hits 2156  Downloads 315  Comment