Video enhancement for increased spatio-temporal resolution in thermal videos: demonstration on a pool fire

Abstract

A spatio-temporal video enhancement of a small-scale pool fire is performed to address the typically low spatial resolution and frame rate of inexpensive infrared (IR) cameras. Improving image quality can increase the applicability of low-cost thermal cameras for certain research tasks and analyses. The spatial resolution and frame rate are doubled, from 310 × 250 pixels (px) to 620 × 500 px, and from 25 frames per second (fps) to 50 fps, as well as from 50 fps to 100 fps. Spatial resolution enhancement is achieved using super-resolution methods based on deep learning, employing several pre-trained models: Fast Super-Resolution CNN (FSRCNN), Efficient Sub-Pixel Convolutional Network (ESPCN), Enhanced Deep Super-Resolution (EDSR), Laplacian Pyramid Super-Resolution Network (LapSRN), and Real-ESRGAN. The footage consists of an n-heptane pool fire recorded using a mid-wave infrared (MWIR) FLIR X6981 HS InSb camera. EDSR provides the best performance for both purely resized images and images subjected to complex degradation. For temporal enhancement, a pre-trained frame interpolation model, FLAVR (Flow-Agnostic Video Representation), is used. The resulting interpolated frames appear realistic and preserve the overall flow direction and shape of the flame. The interpolated frames are compared with ground-truth data to validate the accuracy of the temporal enhancement.