Avoiding "guessing" the transcription

[Jon]

Thanks for this! I see where you’re coming from with dangerously misleading transcriptions. In practice, Leo is guessing every single word of the transcript. For the model, the difference between legibility and illegibility is not binary but scalar. The reason why we don’t want to include an [illegible] sign is because we don’t want to limit the potential scope for the model to learn in the future. To address this issue we’re planning to add confidence metrics. See here:

Transcription failures

I’m not sure why Leo is ignoring signatures. It’s possible it’s an issue in the training data that we need to go back and fix, so thanks for the heads up! I see the logic on [illegible] signs though we decided not to use them as we didn’t want to kneecap the model. Our goal is that Leo will eventually provide not just useable or passable but superhuman transcriptions. What we do plan to add is some kind of confidence metric visible within the web-app, so users can see what’s likely to need corr…