With the ubiquity of multimedia videos, there has been a massive interest from advertising and marketing agencies to provide targeted advertisements for customers. Such targeted adverts are useful, both from the perspectives of marketing agents and end-users or consumers. Our innovative AI powered system, developed in collaboration with Huawei Ireland Research Centre, allows advertising agencies to use powerful media such as video for marketing and publicity, and the users can interact via a personalised consumer experience according to their individual likes and preferences.
In this work, we develop an AI-powered online advert creation system for next-generation publicity. This online system can automatically detect existing billboards in a video, and seamlessly replace the advertisements contained with new ones. The newly generated augmented video provides a personalised user experience for the consumers. This system will be helpful for online marketers and content developers, to develop video content for a targeted audience.
The backbone of our advert creation system is based on state-of-the-art AI techniques from deep learning and image processing. We use a deep-learning based method to localise the position of the advertisement in an image frame. Our solution uses Convolutional Neural Networks (CNNs) in order to extract billboard specific features from the images and be able to detect these at a later stage. We localise the four corners of the detected billboard, using a deep-learning based refinement network. Once the billboard location is detected, we use state-of-the-art integration and blending techniques in order to place a new advertisement into the detected billboard.
As a part of the project, we are also interested in learning candidate placement of objects within a scenes. We have chosen to investigate placing regularly shaped billboard in street view images. We refer our dataset as CASE dataset, that stands for CAndidate Spaces for advErt implantation dataset. The sources of the images in this dataset is Cityscapes dataset comprising street view images. The download link for the annotation and other details is here.
This project gave us the opportunity to develop innovative approaches to create new video content, with the aid of AI. This will greatly impact the advertisement industry, providing an opportunity to place different advertisements in the same original video depending on the target audience. It will also save the thousands of man-hours currently required to manually curate the original videos. We believe that diversifying our developed system to different domains will completely transform the advertisement industry and provide a personalised user-experience for all consumers.
