Future Potential for Crowd-Sourcing Corrections

I’ve really been enjoying Leo and am impressed with its capabilities. I’m sure Leo’s ability to handle things like proper names, especially place names will improve with time. However, Leo is likely to continue to get some things wrong, just as expert paleographers continue to get things wrong because accurate transcribing is often quite difficult and sometimes simply not possible. I’m wondering about possible ways to correct transcription errors after Leo has done most of the hard work, using the great resource that Leo is creating (beyond the model itself): a large community of scholars and paleographers with overlapping skills, interests and goals in one central place. You’ll likely have heard of the early modern wills project at the University of Exeter, where they are using Transkribus to transcribe English wills, and Zooniverse to crowd source voluntary transcription corrections. Could something similar be possible with Leo, and perhaps even incorporated into its funding/profit model in a fair and rewarding way? I’m thinking something along the lines of earning image credits by checking and correcting the completed Leo transcriptions of fellow users (at least those who would like that assistance). This could get transcriptions closer to 100% accuracy, reward people for putting in the time and expertise to do that work, while fostering a collaborative spirit on the site - something along the lines of how people ask for paleography help on twitter and bluesky, but in a systematized and centralized way. This ‘social media’ angle could be quite a light touch, but something collaborative like this could set Leo apart as a service in an increasingly crowded field. Just a thought, and perhaps something you’re already considering along similar lines.

I think this is a really great idea. We’ve discussed incorporating collaboration features (for functionality like that offered by Zooniverse) before. And Jack and I, separately, have discussed allowing users to pay professional palaeographers using the platform. But perhaps this is the best of both worlds.

I’m imagining that the collaboration feature could have two separate functions. First, you could invite other, known users to work on the same documents, so that they can edit them, with version history preserved, etc. And then second, you could offer an open invitation for other users to transcribe a given image or document, in return for so many credits, as you specify, like on Fiverr. What do you think?

2 Likes

Thanks for the response, Jon. I could see the ‘two functions’ idea working for collaborations. I was thinking something along similar lines. My basic pitch would be:

In the more formal and organized case, users could sign up to be ‘Leo palaeographers’ or some such, similar to volunteer moderators. They choose some sort of remuneration for their service, like discount on monthly subscription, more image credits, or more storage etc. People would have different needs. Then you have this pool of experts who can help as and when other users require. They might be able to contribute to the site in other ‘moderator’ type ways, like we see on product message boards, answering questions etc.

Other users could enlist the help of Leo palaeos in ad hoc or sustained ways. Big projects might want to have consistent editing and correction help - something like the Exeter wills Zooniverse project I mentioned. Others would only need help with a single document transcription. Perhaps there could be different packages - the big projects subscribe (paying a bit extra on top of the usual) for consistent help from the Leo palaeos. The ad hoc users pay in some image credits for more limited assistance.

Then there could be a basic and less formal barter system, where image credits could simply be traded among users for paleaography help, etc.

There could also be completely free options, like you mention, to add people onto each others’ projects, specific images or collections etc, so that they might help as and when. This could foster a more straightforward ‘social and communal’ feel to the site.

Obviously the idea is that Leo will be proficient enough so that errors and edits are few. But the reality with palaeography is that some level of uncertainty and interpretation is always there. I do think a secondary benefit of a site like Leo, and something which could be trumpeted, could be the bringing together of like-minded scholars and experts in one place, where collaboration and conversation could be fostered.

1 Like

I love this - it’s absolutely what we want to do with Leo. Thank you Taylor!