Digitally restoring paintings using convolutional neural networks



We digitally restore dirty and varnish-discolored paintings using a convolutional neural network (CNN) trained only on the edges and small sections of the artwork. Our data are images of paintings before and after they were physically treated by a fine art restorer. The training set comes from only 8 × 8 pixel patches obtained from the edges and small areas within the painting. Instead of RGB, we trained our CNN with patches converted into the La*b* colorspace. We built two CNN models with different inputs: one using the L channel (luminance) and the other using the a*b* channels (chromaticity). This approach allowed us to calculate perceptual color difference and generate virtually cleaned images that closely resemble the actual cleaned paintings. Our results were successful, with a mean color error of ∆E = 5.17 from one painting which is perceptible but acceptable in commercial and industrial contexts. Our findings demonstrate that CNN can infer the appearance of a fully restored painting from just less than 10% of the painting area that has been physically cleaned.


Article ID



Poster Session B (Complex Systems, Computational Physics, and Astrophysics)



How to Cite

MJG Narag, J Baumgartner, and MN Soriano, Digitally restoring paintings using convolutional neural networks, Proceedings of the Samahang Pisika ng Pilipinas 42, SPP-2024-PB-17 (2024). URL: