This blog is part of our colourful countdown to the holiday season where we’re celebrating the diversity and beauty of the natural world. In this post, Héloïse Brown and Will Allen of Swansea University discuss research on species recognition signalling across the animal kingdom and explore what this can tell us about Homo sapiens’ interactions with other lineages of archaic human.

Throughout human evolution we know different lineages of hominins met, interacted and reproduced. The exchange of genetic and cultural material between Neanderthals, Denisovans and anatomically modern humans influenced the course of human evolution (and enabled some of 23andme’s customers to proudly claim they are 2.4% Neanderthal).

The specifics of the interactions between lineages of archaic humans remain a topic for imaginative fiction such as the classic movie “A Quest for Fire” (1981) – modern science can only investigate their genetic and archaeological consequences. Nevertheless archaic humans must have made important decisions about whether to trade, fight, socialise or mate with individuals from more or less distantly related lineages.

Were these decisions influenced by whether an individual appeared to be from the same species (a conspecific) or a different species (a heterospecific), and if so, how were these judgements made? When might our ancestors have been more or less accepting of heterospecifics, and what might the consequences of interactions have been? This blog explores what we can learn about these questions by investigating interactions between other organisms alive today.

Organisms’ ability to perceive another organism as being from their own species or another is known as species recognition. Some species have evolved species recognition signals that give the signaller an adaptive advantage by influencing how the receiver responds to the signal. Most of the time this is to make the species recognition task easier or quicker for the receiver, something our research on guenon monkeys has shown. In this group, different guenon species that live in the same place have evolved especially distinctive faces, making it easier to discriminate between members of their own and other species. However, nature is not always so honest and helpful – sometimes signallers trick receivers into responding as if they were members of another species.

When predators get species recognition wrong

A prime example of a misleading species recognition signal is seen in Batesian mimicry, where undefended prey copies the appearance of another species that possesses a potent defence such as a dangerous toxin, hoping to fool predators into taking them off the menu. This often works but only if the disguise is sufficiently convincing. Fooling predators seems to be easier when defended prey have diverse appearances – in these scenarios predators rely on one or two key aspects of appearance to judge whether the prey is from a palatable or unpalatable species, allowing deceptive Batesian mimics to prosper.

Predators frequently sample prey with warning colouration to see if it is palatable or not, which can give predators a delicious meal if the prey is a Batesian mimic or an unpleasant surprise if the prey is unpalatable or toxic. Anyone who has eaten some dodgy fast-food at the end of a night out will know that hunger and a lack of choice strongly influences decisions about what food to sample, and the same is true of predators’ willingness to sample prey with warning colouration.  

By measuring predation rates on artificial prey in environments where models were more or less common, Finkbeiner and colleagues recently showed that the ability of Batesian mimics to fool predators depends on models being relatively common. This shows that the likelihood of encountering a bad meal is also an important influence on predators’ decisions about whether to attack potential Batesian mimics. Thank goodness for food hygiene ratings.

When parents get species recognition wrong

A related situation exists in brood parasite systems. Brood parasitism occurs when one species recruits another species to take over arduous parenting duties. The classic example is cuckoos laying eggs in other birds’ nests, but brood parasitism has evolved in several groups of birds, fish and insects. The consequences for the unfortunate host are drastic; typically it results in the death of their own offspring and a substantial investment of parental energy in another species’ baby. How are hosts so often fooled?

Behavioural and evolutionary ecologists have identified many adaptations that manipulate hosts’ species recognition systems. These begin operating before eggs are even laid in nests; common cuckoo adults are thought to be Batesian mimics of dangerous predatory sparrowhawks, perhaps to dissuade potential hosts from driving them away from their nests. Parasitic eggs often (although not always!) closely mimic the colour and pattern of the host eggs, making it harder for the host to decide whether to reject an egg that might be on its own. Once chicks are born, parasitic chicks sometimes mimic the overall appearance of host chicks. More often parasitic chicks have evolved specific visual adaptations of their hosts such as an especially wide or colourful gape to ensure they are at the top of the pecking order when it comes to parental provisioning. Thus visual adaptations that manipulate hosts species recognition signalling can facilitate success throughout the brood parasitism sequence.

How humans are affecting animals’ species recognition

Anthropogenic changes are altering how animals use and respond to species recognition signals. In a now classic example, Seehausen and colleagues showed how eutrophication of Lake Victoria caused by human activities increased water turbidity, which had a knock-on effect of making it harder for females of the hundreds of species of cichlid fish inhabiting the lake to see the colourful signals used by males, and discriminate between the signals of conspecifics and sympatric heterospecifics. This has resulted in hybridisation and ultimately contributed to the loss of species diversity as two or more species merged into one.

Even when species recognition signalling is an effective barrier to hybridisation in normal circumstances, habitat and population pressures can make it much more difficult to find an acceptable mate, forcing animals to broaden their preferences. There are many well-known examples of this in captive scenarios such as Tigons, Ligers and mules, however there is also evidence of hybridisation occurring in the wild when populations are stressed or depleted, for example between blue and fin whales or red-tailed monkeys and blue monkeys.

Hybridization often results in phenotypic changes, including appearance. This influence on visible traits enables biologists to investigate the consequences of hybridisation following a breakdown in species recognition signalling. For example, in olive and yellow baboons in Kenya, appearance-based evidence showed hybridisation was common and increasing and that this was specifically the consequence of olive and hybrid males migrating into groups of yellow baboons, a result later confirmed by genetic analyses that revealed broad implications for behaviour and reproductive strategy in groups containing hybrids.

So what does research on species recognition signalling across the animal kingdom suggest about Homo sapiens’ interactions with other lineages of archaic humans? We can be fairly confident that individuals had the sensory capacity to detect and recognise differences in appearance between lineages. Whether or not differences were used in behavioural decisions is much less certain.

The evidence from other animals highlighted here includes examples where individuals are exceptionally sensitive to differences in appearance between species (e.g. egg mimicry in brood parasitism), but also examples where decisions about whether to eat, mate or fight seem unaffected by even quite substantial differences in appearance. The examples of how decisions change as hunger levels or anthropogenic pressures vary suggest that most of the time decisions involving species identity depend more on the context of the interaction than details of appearance.

Discover more posts as part of our Colour Countdown series here.