Speech Recognition - Ready for Prime Time?
by Jarred Walton on April 21, 2006 9:00 AM EST- Posted in
- Smartphones
- Mobile
Processor Utilization - Rapid Speech
Accuracy was obviously far lower with the faster, more casual delivery. What happens with processor utilization?
Dictation Processor Utilization
Transcription Processor Utilization
Processor requirements go way up with a faster speech delivery, at least with Dragon NaturallySpeaking. Perhaps it's my lack of enunciation when I'm speaking fast, but obviously you end up with not only lower accuracy, but also longer speech recognition times.
When you consider that our test system was able to keep up with our regular delivery when dictating (barely), the fact that it requires 15.7 minutes on maximum accuracy for an 8:11 length file shows how important "user training" really is. When dictating at 120 to 130 wpm, both software packages are able to keep up. At 146 wpm, maximum accuracy ends up processing only 77 words per minute. When you throw in the increased number of errors, rapid delivery is definitely not the preferred way to utilize either of these packages. If "Natural" means "Fast" to you, you might want to mentally rename DNS to Dragon "Enunciate-And-Speak-Slower" Speaking.
Transcribe mode is also slower, but it doesn't take nearly as big of a hit. Also interesting is that both the transcription mode and the dictation mode manage to max out processor usage with Dragon NaturallySpeaking but live dictation takes 2 to 5 times as much CPU time. Clearly, the full speech UI is putting a decent load on the CPU. That makes sense, as continually trying to determine whether the user has spoken a special command (i.e. trying to access the file menu) would create some overhead. Still, being two to three times slower seems a bit extreme. Of course, keep in mind that during normal writing, rarely can you speak at full speed for several minutes; normally, you'll take frequent pauses to think of exactly what you want to say next.
Accuracy was obviously far lower with the faster, more casual delivery. What happens with processor utilization?
Dictation Processor Utilization
.png) |
| DNS8 Maximum Accuracy |
.png) |
| DNS8 Medium Accuracy |
.png) |
| DNS8 Minimum Accuracy |
.png) |
| MSWord Maximum Accuracy |
.png) |
| MSWord Medium Accuracy |
.png) |
| MSWord Minimum Accuracy |
Transcription Processor Utilization
.png){kind=spacer} |
| DNS8 Maximum Accuracy |
.png){kind=spacer} |
| DNS8 Medium Accuracy |
.png){kind=spacer} |
| DNS8 Minimum Accuracy |
Processor requirements go way up with a faster speech delivery, at least with Dragon NaturallySpeaking. Perhaps it's my lack of enunciation when I'm speaking fast, but obviously you end up with not only lower accuracy, but also longer speech recognition times.
When you consider that our test system was able to keep up with our regular delivery when dictating (barely), the fact that it requires 15.7 minutes on maximum accuracy for an 8:11 length file shows how important "user training" really is. When dictating at 120 to 130 wpm, both software packages are able to keep up. At 146 wpm, maximum accuracy ends up processing only 77 words per minute. When you throw in the increased number of errors, rapid delivery is definitely not the preferred way to utilize either of these packages. If "Natural" means "Fast" to you, you might want to mentally rename DNS to Dragon "Enunciate-And-Speak-Slower" Speaking.
Transcribe mode is also slower, but it doesn't take nearly as big of a hit. Also interesting is that both the transcription mode and the dictation mode manage to max out processor usage with Dragon NaturallySpeaking but live dictation takes 2 to 5 times as much CPU time. Clearly, the full speech UI is putting a decent load on the CPU. That makes sense, as continually trying to determine whether the user has spoken a special command (i.e. trying to access the file menu) would create some overhead. Still, being two to three times slower seems a bit extreme. Of course, keep in mind that during normal writing, rarely can you speak at full speed for several minutes; normally, you'll take frequent pauses to think of exactly what you want to say next.
Facebook
Twitter
RSS
38 Comments
View All Comments
Googer - Saturday, April 22, 2006 - link
BMW 7 series Speech recognition is about 50-75% accurate (my guess) and some users have more luck with it than others.Googer - Friday, April 21, 2006 - link
I think you should re-benchmark these on a system that is not overclocked. Overclocking may have contibuted to errouneous test results. It is possible that some of the benchmarks could have been better on a normal system. Also I am surprised this was not tested on a Intel Syststem. Prehaps one of the programs may benefit from the Netburst Architeture with or with out dual core.Also I would love to download the Dication and Normal Voice wav files, so I can understand the differance between them. Thanks for the article, it came in perfect time; Someone who is handicaped was asking me about this last night.
JarredWalton - Friday, April 21, 2006 - link
I'll see about putting up some MP3s of the wave files -- of course, that will open the door for all of you to make fun of how I speak. LOLIn case this wasn't entirely clear in article, this was all done on my system that I use every day for work. It's overclocked, and it's been that way for six months. I run stress tests (Folding at Home -- on both cores) all the time. I would be very surprised if the overclock has done anything to affect accuracy, especially considering that I did run some tests on a couple other systems that were not overclocked, and basically removed them from this article because they would have simply taken more time to put in the article, and they didn't give me any new information.
It's pretty obvious that neither of these algorithms benefit from multiple processing cores -- HyperThreading, dual core, SMP, whatever. I also wasn't sure how much interest there would be from people in this topic, but if a lot of people want to know how this runs on Intel systems I could go back and look at one. One thing worth noting is that SysMark 2004 does include Dragon NaturallySpeaking version 6.5 as one of the tests. Of course, the results are buried in the composite scores.
JarredWalton - Friday, April 21, 2006 - link
MP3 links available:http://www.anandtech.com/multimedia/showdoc.aspx?i...">http://www.anandtech.com/multimedia/showdoc.aspx?i...
Note that DNS only uses WAV files (AFAICT), but uploading 45MB WAV files seems pointless. Convert them to WAVs if you want to try them with Dragon.
Googer - Saturday, April 22, 2006 - link
Excellant job on the dictation/wav files, you are a very good reader and have a nice clear and concice voice. ;ThumbsUP)stelleg151 - Friday, April 21, 2006 - link
Cool article. I hope that voice recognition continues to improve, for I think it could be incredibly useful for areas like HTPC, or as you said messenging while doing other things (gaming).Zerhyn - Friday, April 21, 2006 - link
Have you ever tried out speech recognition and been underwhelmed? To you yearn to play the role of Scotty and call out..?
PrinceGaz - Friday, April 21, 2006 - link
Yes, that was the first thing I noticed before I even started reading the article. Maybe they used speech-recognition software to enter that.I think they should have an editor (or at least let another contributor read what others have written) who has to approve an article before it goes live as the current number of tyops is unforgiveable ;)
JarredWalton - Friday, April 21, 2006 - link
I'm doing my best to catch typos before anything goes live, but after being up all night trying to finish off this article, I went to post and realized I didn't have a title or intro. So, I put one in using Dragon, but my diction goes to put when I'm tired, as does my eyesight and proofing ability. One typo in a 44 word intro (I didn't proof/edit it at all) isn't too bad for the software. Bad for me? Maybe, but mistakes do happpen. :)johnsonx - Friday, April 21, 2006 - link
One nice thing about Dragon, despite the high CPU utilization shown in the article, is that it will run quite happily with very lowly systems. I have a customer who uses it all day long on PentiumIII-850's with only 512Mb RAM (the max for those particular systems). The heaviest user there recently upgraded to a low-end Sempron64 with a gig of RAM, and he says the overall system is far more responsive (of course), but Dragon's operation isn't radically better; it worked great on the PIII, and works great now.