Thesis Format

Monograph

Predicting and Modifying Memorability of Images

Mohammad Younesi, The University of Western OntarioFollow

Degree

Master of Science

Program

Computer Science

Collaborative Specialization

Artificial Intelligence

Supervisor

Mohsenzadeh, Yalda

Abstract

Everyday, we are bombarded with many photographs of faces, whether on social media, television, or smartphones. From an evolutionary perspective, faces are intended to be remembered, mainly due to survival and personal relevance. However, all these faces do not have the equal opportunity to stick in our minds. It has been shown that memorability is an intrinsic feature of an image but yet, it is largely unknown what attributes make an image more memorable. In this work, we first proposed new models for predicting memorability of face and object images. Subsequently, we proposed a fast approach to modify and control the memorability of face images. In our proposed method, we first found a hyperplane in the latent space of StyleGAN to separate high and low memorable images. We then modified the image memorability (while maintaining the identity and other facial features such as age, emotion, etc.) by moving in the positive or negative direction of this hyperplane normal vector. We further analyzed how different layers of the StyleGAN augmented latent space contribute to face memorability. These analyses showed how each individual face attribute makes an image more or less memorable. Most importantly, we evaluated our proposed method for both real and unreal (generated) face images. The proposed method successfully modifies and controls the memorability of real human faces as well as unreal (generated) faces. Our proposed method can be employed in photograph editing applications for social media, learning aids, or advertisement purposes.

Summary for Lay Audience

Whether it be on social media, television, or smartphones, we see many pictures of faces every day. Historically, faces were intended to be remembered by individuals based on survival and personal relevance. Nevertheless, all these faces do not have the same opportunity to be remembered.

Generally, memorability is defined as an intrinsic characteristic of an image; however, it is unclear what attributes make an image more memorable. To put it simply, the memorability of an image is a number between zero and one indicating how well it can be remembered. In this work, we first introduced new models for predicting memorability of face and object images. These models are deep neural networks that get an image as their input and predict the memorability of the image as the output. Next, we explored the possibility of modifying images to manipulate their memorability scores. This led us to propose a quick method of controlling the memorability of both face and non-face images. This method is based on a group of models called generative adversarial models (GANs) that are capable of producing realistic images. We supported our method with various experiments and applied it to modify the memorability of some faces/objects/scene images. Most importantly, we evaluated our proposed method for real face images too. In our experiments, it was demonstrated that modifying the memorability of real face images was possible as well as modifying the memorability of simulated face images. Our method can be applied to photo editing applications for social media, school learning materials, or advertising.

Recommended Citation

Younesi, Mohammad, "Predicting and Modifying Memorability of Images" (2022). Electronic Thesis and Dissertation Repository. 8483.
https://ir.lib.uwo.ca/etd/8483