I understand that, but keep in mind that most of these bots run off of databases as well that keep track of sites, also keep in mind that it'd not be hard at all to add in custom variables for sites (which I'm sure they also have).
If you think of how they work, it's not that difficult in the first place to make a spambot, let alone set it to have customizations to fool weak captcha's.
Additionally, some use captcha identification scripts to beat some weak captcha's as well.
Now asking to identify an animal, that in itself could be fun yes, but the customizations to a bot could thwart it easily.
And yes, i understand that you think they wouldn't, but I have seen them customize their targeting scripts, on plenty of sites that have 0 relation to eachother and aren't linked together; so it can and will happen if they really want to link from your site.
Keep in mind, it's not just spam to get people to look at sites, it can also be spam that increases hitrankings, so the most popular sites are the better to abuse in this way, so if you have a large site they will take more time to target you.