Efficient and user-friendly visualization of neural relightable images for cultural heritage applications

We introduce an innovative multiresolution framework for encoding and interactively visualizing large relightable images using a neural reflectance model derived from a state-of-the-art technique. The framework is seamlessly integrated into a scalable multi-platform framework that supports adaptive streaming and exploration of multi-layered relightable models in web settings. To enhance efficiency, we optimized the neural model, simplified decoding, and implemented a custom WebGL shader specific to the task, eliminating the need for deep-learning library integration in the code. Additionally, we introduce an efficient level-of-detail management system supporting fine-grained adaptive rendering through on-the-fly resampling in latent feature space. The resulting viewer facilitates interactive neural relighting of large images. Its modular design allows the incorporation of functionalities for Cultural Heritage analysis, such as loading and simultaneous visualization of multiple relightable layers with arbitrary rotations.

@Article{RKGPGBG24,
  author       = {Righetto, L. and Khademizadeh, M. and Giachetti, A. and Ponchio, F. and Gigilashvili, D. and Bettio, F. and Gobbetti, E.},
  title        = {Efficient and user-friendly visualization of neural relightable images for cultural heritage applications },
  journal      = {ACM Journal on Computing and Cultural Heritage (JOCCH)},
  volume       = {17},
  year         = {2024},
  publisher    = {ACM Press},
  note         = {To appear},
  keywords     = {visual computing,cultural heritage},
  doi          = {10.1145/3690390},
  url          = {https://publications.crs4.it/pubdocs/2024/RKGPGBG24},
}