In Video Surveillance one or more security cameras are used to provide video footage of specific scenes/areas of interest. The primary use of such systems is to detect unusual events which may include criminal intent and/or hazardous situations, also known as Video Anomaly Detection (VAD). If an instantaneous reaction to such detections is desired, very often, a human surveillance operator is required to monitor the video footage in real-time and around the clock. This is not an easy task for neither humans nor algorithmic solutions that are based on Artificial Intelligence (AI) because:

• Unusual events are predominantly scarce, unpredictable and context-dependent. Finding a ‘one-shoe-that-fits-all’ solution remains very challenging to date.
• The use of AI in public places and other sensitive areas is very often constrained by very strict privacy-protecting regulations to avoid manipulation and misuse of this technology.

While the introduction of thermal cameras within the context of Video Surveillance is believed to mostly solve the latter, the reduced amount of textural and color information contained in the footage compared to the well-known RGB domain, however, poses a significant challenge on existing Computer Vision solutions that rely on the most-recent advances in AI.

This thesis thus explores the technical opportunities provided by more traditional concepts from the field of Machine Learning to tackle the task of VAD from an ethically responsible angle while moreover paving the way for solutions that are agnostic to the scene, its context, and the given imaging spectrum range. The presented results reveal that the power of an algorithm is independent of its complexity and the richness of the data it processes. These insights can prove to be of significant value to both industrial and academic research in a time in which the success of an AI-based solution is repeatedly determined by the financial resources of its inventors.