The Chaos of Raw Unknowing

Video artwork 6 min 26 sec

This artwork explores trust and artificial intelligence (AI), and the dangers of assuming that AI for computer vision sees as we do, and therefore understands what we understand. Detectron2, created by Facebook AI Research, can identify and locate elements within a scene. It is trained on huge numbers of photographs; mundane scenes from everyday life. We unfairly ask this network to identify elements within artworks from The National Gallery collection. The narration reflects on why this is trivial for humans but a very hard problem for AI.

EXTENDED DESCRIPTION

Machines do not see as humans see. This artwork reflects on computer vision, the branch of artificial intelligence (AI) that seeks to give computers high-level understanding of images. When we watch AI performing computer vision tasks within their comfort zone, it is easy to conclude that they are performing the task just like a human would. When we see their failure modes, we are often baffled by how arbitrarily wrong they can be.

The Detectron2 network was created by Facebook AI Research in 2019. It is a state of the art computer vision network that has many capabilities; one of which is the ability to perform semantic segmentation. We are unfair to this network, asking it to carry out a task that humans find trivial but that represents an unsolved problem in AI. We pan across a series of artworks from The National Gallery collection, and watch it struggle to consistently interpret elements of the resulting images. The network is trained on a huge number of photographs, representing mundane scenes from everyday life. Artworks are not, in general, everyday life. They are ‘out of distribution’, containing historic scenes and unusual juxtapositions of objects, entities and odd perspectives. Perhaps more importantly, the objects and entities are rendered through application of paint and the artist’s lens, texturally divergent from photographic representations. Humans, through a variety of mechanisms that AI has yet to replicate, view an artwork and trivially perform the translation from interpretation to symbol. AIs do not.

Comissioned by Ali Hossaini, National Gallery X and TAS (Trustworthy Autonomous Systems Hub).

Premiered at the TAS conference 'Trusting Machines? Cross-sector Lessons from Healthcare and Security', 1st July 2021 https://www.tas.ac.uk/bigeventscpt/trusting-machines/

Original artwork: The Dream of Human Life, after Michelangelo, The National Gallery collection.