We encourage divers to submit their photos of grey nurse sharks to us at Spot a Shark or through Sharkbook.ai – every image contributes valuable data for our research and conservation efforts. But have you ever wondered what happens after you click ‘upload’? It’s a fascinating journey of image analysis.

Much more than just a quick comparison, the process of image analysis requires both the support of powerful computer algorithms and the careful eye of a human that help us identify individual sharks and contribute to their conservation.

The first step in this process is crucial: can we see the spots? Each grey nurse shark boasts a unique pattern of spots on its flanks, much like our fingerprints. When an image is uploaded, our humans do a review to assess the clarity and visibility of this pattern. Factors like the angle of the shot, the lighting conditions, and whether the flank is clearly visible all play a role in determining if the image is suitable for analysis. When we cannot see enough spots then these photos are listed as “unidentifiable” in the database.

However, when the spot pattern is clear, the real magic begins! We employ a sophisticated suite of algorithms to delve into the intricate details of those spots and compare them against our growing database of known individuals. Currently, we primarily utilize three powerful techniques: the Modified Groth Algorithm, the i3S Algorithm, and the cutting-edge MiewID Algorithm. Let’s take a closer look at each of these fascinating tools:

Modified Groth Algorithm

Imagine using technology originally designed to map the vast expanse of the universe to identify individual sharks! That’s precisely the case with the Modified Groth Algorithm. This algorithm was first developed to compare the positions of stars in images captured by the Hubble Space Telescope. By analysing the invariant properties of triangles formed by triplets of stars, astronomers could precisely align and compare images. Scientists soon adapted this approach for analysing the unique spot patterns of animals like whale sharks, and subsequently, grey nurse sharks. The algorithm essentially identifies the locations of individual spots and then calculates the relative distances and angles between them. These spatial relationships create a unique “map” for each shark. The computer cannot do it alone though and us humans must guide the algorithm to relevant features. Our experienced researchers manually plot where the fins are and where the prominent spots are. From here, the algorithm extracts the coordinates of the visible spots on the flank. It then forms numerous triangles using these spots as vertices and calculates properties that remain constant even if the image is rotated or scaled differently. By comparing these invariant properties to those of sharks already in our database, the algorithm can suggest potential matches. Ultimately, it’s the human who reviews the matches and determines which one (if any) is a match.

i3S Algorithm

The i3S (Interactive Individual Identification System) Algorithm was developed specifically for photo-identification of animals with natural markings. For grey nurse sharks, the algorithm will focus on the location and sometimes the shape of individual spots to help find matches. It can also perform additional analysis that accounts for differences in the angles and body posture in the images. Just like with the Modified Groth algorithm, it’s the human who reviews the computer’s suggested matches and determines which one (if any) is a match.

MiewID Algorithm

The newest addition to our analytical toolkit, MiewID (Matching and Interpreting Embeddings for Wildlife ID), represents the power of modern Artificial Intelligence in wildlife conservation. Unlike the more traditional geometric approaches of Modified Groth and i3S, MiewID learns directly from vast amounts of image data. By analysing countless images of grey nurse sharks, the AI develops an understanding of what makes one individual’s spot pattern unique from another. This learning did not happen overnight. Training MiewID was a significant undertaking, requiring a year to curate hundreds of high-quality images from our database, followed by a dedicated training period for the AI. And the beauty of MiewID lies in its ability to continuously improve. Instead of just looking at the location of each spot, the MiewID algorithm learns complex visual features and creates a unique numerical “embedding” or digital fingerprint for each shark image. As our database of identified sharks grows and more images are analysed, the AI refines its understanding of grey nurse shark spot patterns, leading to increasingly accurate and efficient identifications. At sites like Bushrangers Bay, where the sharks are typically juvenile with very obvious spot patterns MiewID is the best algorithm. However, to date this algorithm struggles with adult sharks where the spot patterns are either fading or stretched (e.g. for a pregnant shark) compared to when that same shark was younger. This is why it is unlikely we will be able to retire the older spot mapping processes yet.

Human-Computer Partnership

 

Whilst the computer technology is fantastic and ever improving, the computer is not yet able to do this task without the help of a human eye. There is a clear teamwork between human and computer in this spot mapping process. Imagine if a human had to review every image against 18000 other images in the database?! This is clearly where the computer has the strength – the computer can review thousands of images quickly and proposing the top matches in a few minutes. That same task for a human could take months!

But what we have found is that the computer vision technology is not yet as good as the human eye. See the below example of two images that the computer struggled to match. A human can see these sharks are a match, but the computer did not present this as its first choice.

The spot mapping process therefore includes a number of different processes. No single algorithm (or brain) has nailed it…yet. But the world of AI is constantly evolving, and so too are the algorithms we use to protect our marine life. Every year brings advancements in deep learning and computer vision, promising even more sophisticated and accurate methods for individual identification. This means that the more images we collect and analyse, the better our ability becomes to understand and safeguard the populations of these incredible sharks.

So, the next time you submit a photo, remember the intricate process unfolding behind the screen. Your contribution plays a vital role in not just supporting shark research, but in teaching our AI to ever improve in computer vision!

If you want to see these algorithms in action please click here to watch a quick video.