Speech Recognition - Ready for Prime Time?
by Jarred Walton on April 21, 2006 9:00 AM EST- Posted in
- Smartphones
- Mobile
Speech Accuracy - Precise Dictation
Moving to a more typical speech pattern and not my stilted dictation voice, how do things change? First, this sound file is 1208 words and 8:11 in length. This is obviously a much more rapid delivery, typical of more casual conversation. At 147 wpm, this is faster than all but the best typists can achieve. The question is, how will the software deal with the faster delivery? We'll start with dictation mode again.
Dictation Accuracy
Accuracy drops several percent in both products, but DNS is clearly affected more. In fact, at just over 90% accuracy, there are enough errors that it's questionable whether or not you'd really be saving any time. In this particular instance, Microsoft's speech engine makes half as many mistakes. In a rather odd twist, both products also get more accurate as you turn their accuracy gauge down. It appears that with rapid text, the algorithms almost "overthink" the waveforms.
Transcription Accuracy
Switching to transcription mode, Dragon once again comes out on top. What's most likely happening is that Dragon is trying to determine whether or not you're issuing commands or dictating text, and it ends up wasting time that could be better spent analyzing your speech patterns. Basically, Dragon has a more complex user interface, and in straight rapid dictation this appears to be a handicap. Trascribing a WAV file shuts off many of the extras, so performance and accuracy improve substantially.
Moving to a more typical speech pattern and not my stilted dictation voice, how do things change? First, this sound file is 1208 words and 8:11 in length. This is obviously a much more rapid delivery, typical of more casual conversation. At 147 wpm, this is faster than all but the best typists can achieve. The question is, how will the software deal with the faster delivery? We'll start with dictation mode again.
Dictation Accuracy
Accuracy drops several percent in both products, but DNS is clearly affected more. In fact, at just over 90% accuracy, there are enough errors that it's questionable whether or not you'd really be saving any time. In this particular instance, Microsoft's speech engine makes half as many mistakes. In a rather odd twist, both products also get more accurate as you turn their accuracy gauge down. It appears that with rapid text, the algorithms almost "overthink" the waveforms.
Transcription Accuracy
Switching to transcription mode, Dragon once again comes out on top. What's most likely happening is that Dragon is trying to determine whether or not you're issuing commands or dictating text, and it ends up wasting time that could be better spent analyzing your speech patterns. Basically, Dragon has a more complex user interface, and in straight rapid dictation this appears to be a handicap. Trascribing a WAV file shuts off many of the extras, so performance and accuracy improve substantially.
Facebook
Twitter
RSS
38 Comments
View All Comments
JarredWalton - Friday, April 21, 2006 - link
That's definitely true -- if you look at how accuracy scales with CPU usage, doubling and even tripling the processor time comes with only incremental increases in accuracy. I do have to say that I noticed it being a little sluggish on my single core system when I was multitasking, but obviously I push my computers a little harder than a lot of people. Depending on what you're willing to live with in terms of speed, I'm sure both Dragon and Microsoft speech recognition can work on a Pentium III level system.LanceM - Friday, April 21, 2006 - link
So is that selection typical Asimov? If so, it has convinced me to never bother reading any of his works.His ideas/plots/etc. may be interesting, but I don't think I could handle phrases like, "as if she were some dried-up, old-maid teacher." Give me Joseph Conrad or William Faulkner.
Dfere - Monday, April 24, 2006 - link
Asimov is classic Sci-Fi- pulp, which usually had a gritty detective-novel appeal. Hs works are in large part murder mystery type novels. You have to understand the nature of the literature, the history and the author. I don't think a critique is deserved until then.Most Sci Fi writers of any ability first master imaginative concepts and apply them, even Drke and Sirling.
I give Kudos to the staff for including literary comments, the poster who said this should not be a book of the month club lives a very one dimensional life.
Shoal07 - Friday, April 21, 2006 - link
What makes Asimov special is many of his ideas in sci fiction are comming true today or are atleast on the horizon. Asimov shaped the way many of us picture the future.goinginstyle - Friday, April 21, 2006 - link
Why does the Anandtech staff revert to literary quotes in their reviews now? This is a computer website, not a book club.JarredWalton - Friday, April 21, 2006 - link
I read Asimov's foundation series as a teenager, and I loved it. He gave me lots of fanciful dreams about where technology might go in the future, and even though some of the writing styles have changed over the years, I still find a lot of these old sci-fi books to be entertaining. You should try reading War of the Worlds if you think that quote was bad. LOLSorry if some of you didn't like the quote. Everyone has their own dislikes and likes, but in the end it's just an introduction. I hope to one day be able to yell at my computer and have it properly understand what I say, as well as the context (i.e., yelling means something is going wrong, and maybe it can help me out). Will we ever get there? Probably some day, but whether it happens in our lifetimes or not is anyone's guess.
NegativeEntropy - Saturday, April 22, 2006 - link
I like the use of quotes -- though it does remind me a bit of being in English/writing class ("Always do something in the introduction to get your audience's attention...").On the subject of "classic" Sci-fi writers, I also still enjoy old school Heinlein. Though his characters can get a bit repetitive across his pile of works, many of the science ideas are still valid (and I share much of his apparent personal philosophy).
On the actual article -- thanks for doing it. I have been curious where this technology was at in terms of every day usage and hardware requirements.
Regarding CPU usage, it's possible DNS attempts to use whatever resources are available based on preferences. i.e. on minimum, it attempts to impact the system minimally, regardless of the CPU resources available; say 25% on min, 50% on med and 95% on max with the percentage staying relatively consistent from a P3 1GHz to an A64 2.6GHz. This would explain its reported good scaling from system to system. If you want to test it, underclock your A64 system to half its frequency and compare utilization at the medium setting.
kristof007 - Friday, April 21, 2006 - link
Here at Anandtech you can always count on to find something else. Great article! I tried out speech recognition a few years back and I got frustrated with it over one thing or another so I just dropped it and went back to typing. I've been typing for about 8 years now. I never learned the "proper" way to type where every finger has a spot. Anyway I hope Vista will make speech recognition WAAY better so that it could be used around the OS AND for speech recognition.Thanks for the article!