This article sets out to demonstrate the uneven history of settler-Australians’ labelling of Indigenous cultural objects and documents as ‘art’. Using the case of William Barak (c. 1824–1903) as its example, it asks, how was Barak’s work understood prior to the major re-evaluations of Aboriginal art as ‘art’ in the 1980s? A series of fleeting moments of understanding, exchange and recognition provide a hitherto-overlooked genealogy of the shifting reception of Barak’s paintings and drawings within his own lifetime and up to the 1940s. These moments encompass his agency in diplomatic exchange, his peer-to-peer relationships in Melbourne’s colonial artworld, and the early placement of Barak’s work in cultural institutions leading eventually to the first inclusion of his work in an art exhibition in 1943. Selected examples from this trajectory demonstrate an uneven path to recognition while illustrating their ability to exceed the category of art from a western viewpoint.