Could AI make awards judging fairer in the future?
Introducing Pearl, the first AI jury experiment. To celebrate its 10th anniversary, the Belgian MIXX awards has asked the first artificial intelligence jury president, called Pearl, to designate the winner of a special award during this year’s MIXX Award gala. With Pearl, the organizers want to open a debate about the future of award judging, particularly in light of some recent criticism. Is it possible that AI could make judging fairer? Could it help make creativity measurable? And how might it complement human juries?
Is creativity measurable?
While award shows normally just look at the future of advertising, this time the MIXX awards have decided to look at the future of judging! This first AI experiment on judging will investigate if creativity is measurable and how AI might be able to help the judging system. Named after Radia Perlman, one of the prominent female Internet pioneers, the result is Pearl.
More or less measurable
Pearl was fed with all international MIXX case studies from the last decade and given 3 months to analyse the texts, videos, results and even music making her surely the most experienced advertising jury on the planet. Pearl uses state-of-the-art technology, for example: Microsoft gave early preview access to their Video Indexer service, which extracts insights from videos using Artificial Intelligence. She has taught herself to think like humans and has already reached an accuracy level of about 76%. In other words, if you feed her a new case, she is pretty good at being able to predict whether it will be a winner or not, and even what colour of award you might expect. “This is beyond our expectations”, says Jos Polfliet, Head of Machine Learning at Faction XYZ, “but it seems that creativity is more or less measurable after all.”
What Pearl learns from us:
- Pearl is not impressed by the number of impressions you ran. She prefers qualitative metrics such as ‘trending topic’ and ‘earned media’.
- If your campaign has had more than 100.000 shares, don’t hesitate to share it with the jury.
- Being mentioned by dedicated ad press is no guarantee to win. Being mentioned by BBC & CNN however, does give more credibility to your case.
- Upbeat feel good music is so much overused that it’s being perceived as cliché and generic.
- The use of a voice-over has a positive effect on the outcome.
- When it comes to new technologies, buzzwords have the most success in the year they were launched, and even more success in the year afterward.
- Case movies of winning campaigns contained 25% more suits than non-winning cases.
- Pearl can identify the originality of your campaign, but is still learning to recognize if an original idea is also meaningful.
There is room for improvement
Pearl is already giving us great insights into what is important in an award winning idea. She has made links between narrative styles and awards, between choice of instruments and awards, and she also has understood that the number of impressions doesn’t mean that much, while number of shares is far more important.
There is room for improvement though, as she needs to be able to better recognize humour and a creative idea. On the plus side, she can detect original combinations, which helps, “but sometimes she has the reasoning of a pregnant woman,” says Peter Ampe, ECD of DDB in Brussels. “She thinks Tuna with whipped cream sounds like a great idea! But she’s still learning and her accuracy will continue to improve.”
Back-up for pre-judging
Pearl is not quite ready yet to replace human jurors, but she can already complement them. She doesn’t of course have the flaws human juries have as she is able to give every case the same level of attention, never grows tired, and perhaps most importantly doesn’t have friends to defend or a network or country to promote.
There is no doubt AI will be able to help award shows in different ways in the future. For example, in larger award competitions, where much of the pre-judging is done in small groups, it is sometimes the case that good campaigns are overlooked. With AI, it will be possible to judge which campaigns might be potential winners and then compare this with the shortlist the human jury has put together. If some of these campaigns are not shortlisted, the human jury could perhaps look at these campaigns again and maybe reconsider their judgement.