Google  
Home
   
e-ASRNews Log-on
     
Subscription Info
 
Market Reports 
 
Speech Industry Links
 
Examples of Good VUIs
   
Repairs for wrecks
   
Gethuman Consulting Services
  
Tern Systems
 
Get2Human

Repairs for Wrecks 
NYC Trip Planner  Milwaukee Journal Sentinel  Commonwealth Choice   NSTAR    CMP    US CIS  AARP  IRS  The Annual Credit Report Service  American Airlines   Tufts   Charter   Washington Post   NY Times   WSJ   SSA   Dell   Virgin Mobile    VerizonBroadband   Cingular   Apple  Federal Trade Commission  Eureka   FedEx  Comcast    United Airlines   Verizon    CareMark   Amtrak   USPS   aol   Bank of America  

   
Telephone self-service is able to provide real value to consumers. It makes access to information available 24/7. It is available without waiting in line. Consumers can purchase things. They can check availability. They can check the status of an order. They can obtain directions. In addition to making customers happy, telephone self-service is also able to save money for the enterprise that is providing it.

Unfortunately, this situation is rarely achieved. Telephone self-service is intensely disliked by many consumers because it is implemented very badly. The caller wants a quick & easy experience and most telephone self-service instead provides an experience that is slow & difficult.

We present here a few of these telephone self-service implementations that were evaluated during the last months. These were selected mostly at random and it's unlikely that they represent the worst implementations.

Table 1 summarizes the results of this testing. We have adapted the www.get2human.com Voice User Interface (VUI) Standard as the basis for our evaluation. The scoring/rating scheme is simple to understand and scoring of a particular VUI is relatively easy to do. All that is needed is to make a phone call and observe how the system responds. The scheme that we are using has some obvious limitations. The fact that we give each item equal weight is certainly arguable. On the other hand, despite its simplicity, it appears to yield results that seem to correlate with the relative goodness of the self-service system.

For each of the VUIs that we tested, we provide a discussion of it from the perspective of the caller. This is done by Walt Tetschner, who has suffered through many poorly implemented telephone self-service implementations. We also provide a clinical analysis of the system from the viewpoint of a highly-regarded VUI designer (Dr. Walter Rolandi). This identifies what the system is doing incorrectly and offers recommendations regarding what should be done to repair it.

Table 1

Get2Human Telephone Self-service Rating
Get2Human item NYC Trip Planner Milwaukee Journal Sentinel Common-wealth Choice Dell SSA WSJ NY Times Washington Post Charter Commun-ication Tufts Health
care		 American Airlines The Annual Credit Report Service IRS AARP US CIS NSTAR Central Maine Power (CMP)
The caller must always be able to dial 0 or to say "operator" to queue for a human. 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 1
An accurate estimated wait-time, based on call traffic statistics at the time of the call, should always be given when the caller arrives in the queue.  A revised update should be provided periodically during hold time. 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 NA NA
Callers should never be asked to repeat any information (name, full account number, description of issue, etc.) provided to a human or an automated system during a call. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
Callers should always be offered the option to be called back when a human is not available. This option should be offered periodically during hold time. If 24 hour service is not available, the caller should be able to leave a message and be called back promptly when humans are available.. 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0
Speech applications should provide touch tone (DTMF) fall-back, where appropriate.  0 1 NA 0 0 0 NA 1 NA 1 1 1 NA 1 NA 1 NA
Callers should not be forced to listen to long/verbose prompts.  0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0
Callers should be able to interrupt prompts (via dial-through for in DTMF applications and/or via barge-in for speech applications) whenever doing so will enable the user to complete his task more efficiently. 0 0 0 0 0 0 1 1 0 0 1 0 0 1 0 1 1
Do not disconnect for user errors, including when there are no perceived key presses (as the caller might be on a rotary phone); instead queue for a human operator and/or offer the choice for call-back.  0 1 0 1 0 0 0 1 1 0 1 0 0 1 0 1 1
Default language should be based on consumer demographics for each organization.  Primary language should be assumed with the option for the caller to change language.   (i.e. English should generally be assumed for the US, with a specified key for Spanish.) 1 1 0 1 0 1 1 1 0 1 1 1 0 1 0 1 1
All operators/representatives of the organization should be able to communicate clearly with the caller  0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1
Gethuman Subtotal (Average score) .10 0.60 0.11 0.20 0.10 0.20 0.33 0.70 0.33 .30 0.6 .30 0.11 0.90 0.11 0.78 0.75
Recognize me by caller ID (home or mobile) and then let me enter a four digit PIN to authenticate, so you then have my info on file and don't make me type in long account numbers etc. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
No "Your call is important to us" greeting.  1 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1
No "You can access our website to answer most questions" message. I already know that. 0 1 0 0 0 0 1 1 1 1 0 0 0 1 0 1 0
No "Please listen carefully, since our menu has changed" message.  1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 1 0
Do not play advertisements to me 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Do not speak to me in legalese or corporate language 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1
Is it a Quick-&-Easy experience? -3 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0
Understand me when I speak a natural number 0 1 NA 0 0 0 NA NA NA NA NA NA 0 NA NA 0 NA
Total Score (Average) 0.00 0.61 0.31 0.33 0.28 0.28 0.50 0.65 0.50 0.35 0.59 0.41 0.24 0.88 0.19 0.76 0.60
 
Get2Human Telephone Self-service Rating (continued)
gethuman item BofA AOL USPS Amtrak CareMark Verizon United Airlines Comcast Eureka FedEx FTC Apple Cingular Verizon Broadband Virgin Mobile
The caller must always be able to dial 0 or to say "operator" to queue for a human. 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
An accurate estimated wait-time, based on call traffic statistics at the time of the call, should always be given when the caller arrives in the queue.  A revised update should be provided periodically during hold time. 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0
Callers should never be asked to repeat any information (name, full account number, description of issue, etc.) provided to a human or an automated system during a call. 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
Callers should always be offered the option to be called back when a human is not available. This option should be offered periodically during hold time. If 24 hour service is not available, the caller should be able to leave a message and be called back promptly when humans are available.. 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
Speech applications should provide touch tone (DTMF) fall-back, where appropriate.  1 1 1 1 1 0 1 1 0 1 0 0 NA 1 1
Callers should not be forced to listen to long/verbose prompts.  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Callers should be able to interrupt prompts (via dial-through for in DTMF applications and/or via barge-in for speech applications) whenever doing so will enable the user to complete his task more efficiently. 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0
Do not disconnect for user errors, including when there are no perceived key presses (as the caller might be on a rotary phone); instead queue for a human operator and/or offer the choice for call-back.  1 0 0 0 1 0 1 1 0 1 0 0 0 1 0
Default language should be based on consumer demographics for each organization.  Primary language should be assumed with the option for the caller to change language.   (i.e. English should generally be assumed for the US, with a specified key for Spanish.) 1 0 1 1 0 0 0 0 1 1 0 1 0 1 0
All operators/representatives of the organization should be able to communicate clearly with the caller  1 0 1 1 1 1 0 1 1 1 1 1 1 0 0
Gethuman Subtotal (Average score) 0.50 0.10 0.40 0.40 0.30 0.10 0.20 0.40 0.20 0.60 0.10 0.30 0.11 0.30 0.10
Recognize me by caller ID (home or mobile) and then let me enter a four digit PIN to authenticate, so you then have my info on file and don't make me type in long account numbers etc. 0 0 0 0 0 0 0 1 0 1 0 0 0 0 NA
No "Your call is important to us" greeting.  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
No "You can access our website to answer most questions" message. I already know that. 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1
No "Please listen carefully, since our menu has changed" message.  1 1 1 1 0 1 1 1 1 1 0 0 1 1 1
Do not play advertisements to me 0 0 1 1 1 0 1 1 1 1 1 0 0 0 1
Do not speak to me in legalese or corporate language 0 0 1 1 0 1 1 1 1 1 0 1 0 0 0
Is it a Quick-&-Easy experience? 0 0 1 0 0 0 0 1 -7 1 0 0 0 0 0
Understand me when I speak a natural number 0 1 0 0 0 1 1 NA 0 1 1 NA 0 1 0
Total Score (Average) 0.39 0.22 0.56 0.44 0.33 0.28 0.39 0.59 0.00 0.72 0.22 0.29 0.18 0.33 0.29

6/11/09

Walt and Walt’s VUI View: NYC Transit Trip Planner

The Caller’s View from Walt:

I called the NYC Transit Trip Planner Voice to see how well it worked.  I had seen a lot of recent hype re the new speech-enabled planning that Nuance and Aspect Software had implemented.

I called 718-330-1234.  After a welcome greeting that was too long, I encountered a menu that used a different voice for each menu item.  It had long pauses after each item.  It only had 5 menu items, but still managed to use 1 two times.  The response to my entries was inconsistent.  Sometimes it ignored the actual input (no overstrike).  Other-times it provided the information even though it wasn’t requested. (it seemed to insist on telling the caller about the re-routing of buses on Broadway, independent of the key that I pressed).  Getting to the voice service was problematical.  Pressing 9 to “Continue” was a menu option.   This seemed to get me to the Trip Planner Voice, but you had to patiently listened to all of the menu items (over a minute).    

The speech system was a highly-directed implementation.  It explained in much detail how I could specify my departure and destination points, times, and modes of tramsportation.  Huge delays between a caller input and a subsequent response.  I was calling from a quiet office environment and it had lots of speech recognition problems.  Classical error recovery torture.  It asked me when I wanted to leave and I said “right away”.   It had difficulty recognizing this and forced me to specify a precise time.   Unnecessary confirmation of my responses.    It finally read the travel instructions to me.  It had part of each phrase spoken by the same voice and used TTS for only the variable portion.  This would have worked better if they used the TTS voice for the entire phrase, since the switching of voices is a distraction.

I indicated that I wanted to travel from The Empire State Building in Manhattan to The Yankee Stadium in The Bronx.  It was 10:30AM when I called.  The system indicated that the next subway train would not leave until 11:43AM.   This seemed strange – an hour-&-a-half wait on a busy weekday morning.  I decided to talk to an agent.  The automated system did not tell me how long the wait would be but did state that an agent would be with me in “just a moment”.  After waiting for 3-4 minutes and being told to go to the MTA website a number of times, I finally reached an agent.  No functioning CTI was in place and I had to repeat everything that I had entered into the IVR.  The agent indicated that the departure time for the train that I was looking for at 10:30AM, was 10:43AM and that the 11:43AM time that it gave me was incorrect.  I went to the NYC Transit web site www.tripplanner.mta.info   It confirmed that the departure time was 10:43AM

This is one of the very worst self-service systems that I’ve run into.  Virtually everything about the implementation seems to be wrong.  In addition to it being an irritating and time-wasting experience, the information that it provided was wrong.

 

“Dr.” Walt Tetschner examination of NYC Transit Trip Planner:

Patient:                       NYC Transit Trip Planner

Task:                                      Get from The Empire State Building to The Yankee Stadium

Toll Free Number:    1-718-330-1234

 

The Examination Session

Several calls were made to the IVR on a busy weekday morning.   Only one call was actually necessary to complete the task. Call duration was 298 seconds or approximately four and a half minutes.

Call Details:

The “system” appears to be two or more systems linked together.  The call was initially answered by a DTMF system.  A menu was then presented that used a different voice for each menu item.  Each menu item was followed by a long pause (approximately 5-6 seconds), which seemed to be giving the caller adequate time to respond.  If the caller did not respond with a key depression within this pause period, the system appeared to assume that this item was rejected and proceded to the next menu item with a different voice. 

Upon pressing DTMF9 at the last item of the “menu”, the call is apparently transferred to another system where yet another voice talent appears.  At this point, the “system” switches over to speech recognition.  A significant delay occurs to get switched to the speech system.

After a long comment about wheelchair accessibility, it finally says:

“If you need to know how to get from one location to another, say plan a route.”

 I said: “Plan a route.”

 “All right. In order to plan a route, I need to know what type of starting point, your destination and when you will be traveling.”

“Which borough are you starting in?”

I said: “Manhattan.”

 “Sorry I didn’t get that. Are you starting in Manhattan, Brooklyn, Queens, the Bronx, Staten Island?”

 I said: “Manhattan.”

 “I need to know which type of starting point you will be using. So, would you like to tell me an intersection, landmark, or point of interest, subway stop, or address?”

I used the landmark approach going from The Empire State Building to Yankee Stadium.  This was a tedious experience with long prompts, long pauses, and frequent speech recognition errors, where I had to repeat the input.

After it provided the travel information to me, it informed me that I could say “goodbye” and hangup to end the call.

The Diagnosis

1.      The “menu” that sequentially reads each item and waits for a response, is a grossly inefficient way to communicate with a caller. 

2.      The “system” mixes exclusive interactive modalities (DTMF only and speech only).

3.      The database access is providing incorrect information.

4.      System uses different voice talents for different phrases or portions of a phrase.

5.      Long pauses exist everywhere.

6.      A long web ad is played.

7.      Many prompts are unnecessary or far too long.

8.      The speech system seemed to have difficulties recognizing simple, easy to recognize utterances.

9.      The system stupidly apologizes to the user when it fails to “understand”.

The Prescription

1.      The existing “menu” structure should be eliminated in favor of a true menu.  Better yet, a NL implementation with agent-assisted ASR (AAA) would yield a superior solution.

2.      Adopt a single interactive modality universally supporting both DTMF and speech.

3.      Fix the database access to assure that the data that is being delivered is accurate.

4.      Add CTI and make sure that the agents are using it.

5.      Add a queue wait time announcement.

6.      Implement a consistent overstrike.

7.      Either shorten or eliminate the web ad.

8.      Use a single professional voice talent for all prompts.

9.      Eliminate or trim any unnecessary prompts and prompt content.

10.  Eliminate apologies (users don’t care when computers are “sorry”).

11.  Investigate and tune system grammars.

The Prognosis

Good if the prescription above is followed.  The problems are so severe though, that this requires virtually a total re-design of the existing system.

 

 

 

Jan 8 2008

Walt and Walter’s VUI View: Milwaukee Journal Sentinel

The Caller’s View from Walt:

I called the Milwaukee Journal Sentinel to obtain information about starting a subscription. I was greeted by the typical up-front verbage that wasted my time by forcing me to listen to bunch of unnecessary stuff like instructing me to use the MJS website.  Finally gave me a TouchTone menu, but managed to change voices to cause confusion.  I selected something that was vaguely close to what I was calling about which put me into another system that had a different voice and required speech input.  I responded to the “Something else” category by saying “new subscription”.  When requested, I entered my phone number and zip code and it came back and gave me the rate for a 1-week subscription.   The rate was $18.42 a week.  It then started to verify my name and address by spelling everything out.  The verification process that they use is painful.  Virtually everything that you say gets a repeated back to you followed by: “is that correct?  Please say yes or no?  Half of the time it does not recognize the yes response properly and you have to repeat it.  I wanted rates other than the weekly rate which seemed rather high.  The automated system didn’t appear to have this information available so I attempted to get to an agent.  After hitting 0 a few times, I was connected into a wait queue that informed me that the weather was bad and negatively impacting newspaper deliveries and that 19 people were ahead of me in the line.  It offered to call me back.  I opted for this.  After a few days, I have still not received a call-back.  I attempted to use the system again.  I entered all of my information again and then waited patiently in line.  When I finally got to the agent, she did not have any of the information that I had laboriously entered into the automated system. 

Dr. Walter Rolandi examination of Milwaukee Journal Sentinel Customer Service:   

Patient:                       Milwaukee Journal Sentinel

Task:                           Determine the cost of a new subscription.

Toll Free Number:    1-800-759-6397

 

The Examination Session

Several calls were made to the IVR over a holiday weekend.   Only one call was actually necessary to complete the task, which, if unnecessarily time consuming, was reasonably easy to complete.  Call duration was 147 seconds or approximately two and a half minutes.

 

Call Details:

The “system” appears to be two or more systems linked together.  The call was initially answered by a DTMF system.  After reading a 30-second series of introductory messages which included a web ad, the user is instructed to “press one for our automated phone system”.  These various messages were apparently recorded using three different female voices, one decidedly more professional sounding than the others.

 

Upon pressing DTMF1, the call is apparently transferred to another system where yet another voice talent appears.  At this point, the “system” switches over to speech recognition.

 

After an approximately 10-second banner prompt explaining that inclement weather had caused various delivery problems, the system says:

 

If that’s not why you’re calling, which of these can I help you with?

A delivery issue

Vacation hold

Your account or

Something else.

 

The system then immediately repeated these choices.  While the “Your account” option seemed like a possibility, I choose the “Something else” route.  This quickly led to a four-item submenu, the second choice of which was “Start a new subscription”.

 

After saying, “Start a new subscription”, the system prompted me for my telephone number.   It failed to recognize the number on the first attempt.  After the second attempt, the system read my number back correctly saying:

 

Is that right?

Just say “yes” or “no”.

 

The system then prompted me for my zip code, again asking me to confirm what it “heard” with:

 

Is that right?

Just say “yes” or “no”.

 

Oddly, it failed to recognize me when I answered, “Yes” but got it on the re-prompt.  About ten seconds of information about a payment plan was then read out, finally followed by the subscription cost amount.  Upon hearing the rate, I hung up.

 

The Diagnosis

1.      The “system” mixes exclusive interactive modalities (DTMF only and speech only).

2.      System uses two, three or four different voice talents.

3.      The web ad is an IVR cliché.

4.      Many prompts are unnecessary or far too long.

5.      The system stupidly and repeatedly instructs the user to answer its yes/no questions with, “Just say ‘yes’ or ‘no’”.

6.      While the production quality of the speech-only component is very good, the overall production quality not very professional.

7.      The speech system seemed to have difficulties recognizing simple, easy to recognize utterances.

8.      The system stupidly apologizes to the user when it fails to “understand”.

 

The Prescription

1.      Adopt a single interactive modality universally supporting both DTMF and speech.

2.      Either shorten or eliminate the wed ad.

3.      Use a single professional voice talent for all prompts.

4.      Eliminate the pointless “Just say ‘yes’ or ‘no’” prompt.

5.      Eliminate or trim any unnecessary prompts and prompt content.

6.      Eliminate apologies (users don’t care when computers are “sorry”).

7.      Investigate and tune system grammars.

 

The Prognosis

Really quite good.

 

 

 

 

 

Dec 8 2008

Walt and Walter’s VUI View: Commonwealth Choice

The Caller’s View from Walt:

Commonwealth Choice is the government coordinator for the universal healthcare program that the state of Massachusetts offers to their residents.  I called their customer support line (877-623-6765) to find out what the basic eligibility requirements were. 

Commonwealth Choice starts by playing an ad and then telling you to go to their web site, describes what it contains and what the address is.  It then instructs the caller to press 1 for English.  It then forces the caller to listen to a long menu of choices.  If you press 5 at the 1st menu, you are then requested to indicate the number of adults and children in your family and then checks that your income is below the cut-off.  If you qualify, it then gives you another long menu.  If you press 0, it places you in a wait queue for a human.  The wait time typically appears to be in excess of 5 minutes.

I noticed that the number of people in your family can’t be more than 8.  If you enter 9 (or any double digit number), you are shuffled back to the main menu.   This looks like a glitch in their logic, since families with more than 8 members do exist and would appear to be eligible for State-provided health insurance.

After entering my family information and my income, I was connected to a human (after a wait of about 5 minutes).  During the wait, the system asked me for additional information regarding what I was calling about.  When I finally did get to an agent, she was polite and seemed knowledgable.  None of the information that I had provided to the system was passed to the agent and I had to start all over again. 

 

Dr. Walter Rolandi examination of Commonwealth Choice Customer Service:          

Patient:                                  Commonwealth Choice

Task:                                     Determine eligibility for universal healthcare plan.

Toll Free Number:              1-877-623-6765

 

The Examination Session

Technically, it was not possible to know whether the task was actually complete.  After I answered a few questions, the system transferred the call to a representative, presumably because my answers implied that I was not eligible for the plan.  Only one call was required to navigate to this point and doing so required 124 seconds to complete.  The entire call lasted 133 seconds. 

 

Call Details:

The system is a highly structured DTMF only application.   The user experience begins with a brief greeting followed by about 20 seconds of pointless introduction and a web ad.  Then follows a 14 second Spanish option.  After selecting “English”, an extremely wordy 6-item, 46 second Main Menu is heard.   DTMF choice 5 began with:

 

If you are unsure whether you are eligible….

 

I entered DTMF 5.   The system then asked for the number of adults and children living in the household.  It apparently uses this number to calculate the household income breaking point for the number of people one enters.  The system then asked a yes/no question as to whether the household income exceeded a particular amount.   After answering, “yes”, I heard the transfer message.

 

Thus, the system required over two minutes to obtain three simple bits of information and transfer the call.

 

The Diagnosis

1.      System prompts are far too wordy.

2.      IVR clichés such as a web ad and a Spanish option are present.

3.      Many prompts are clearly unnecessary.

4.      It requires far too much time to perform a presumably simple sorting process.

5.      Overall production quality is unprofessional.

6.      Production quality of the concatenated income amount is all but embarrassing.  It uses mixed male-female voices and obviously inappropriate intonation.

 

The Prescription

1.      Get rid of the wed ad and Spanish option.

2.      Chop the Main Menu to three, succinctly stated items.

3.      Use professional voice talent.

4.      Use the same voice talent for all prompts.

5.      Eliminate or trim down any unnecessary prompts and prompt content.

 

The Prognosis

Probably all right.

 

 

Nov 6 2008

Walt and Walter’s VUI View: NSTAR

The Caller’s View from Walt:

I called NSTAR to obtain information on my account balance.  NSTAR has implemented a speech-enabled attendant.  It worked pretty well.  It asked me what I was calling for and it recognized my saying account information.  It then took my account number.  This worked fine is I said each digit individually but did not recognize it when a said it as a natural number.  When it didn’t recognize the account number, it defaulted to Touch-Tone right away.  This is a much better approach than the “did you say ….” or asking you to say it over again.   I obtained my account balance in well under a minute.  I had a question about it and requested to talk to a human.  I was connected to a human in less than a minute.  She did not have my account number available.  NSTAR has implemented a pretty good self-service capability.  It provides a quick-&-easy experience to the caller.  They then spoil it all by not passing the information to the agent.  Whatever time is saved by having a quick-&-easy implementation is now a lost by having to repeat everything to the agent. 

I attempted to talk to a human after normal business hours.  I was informed that they were closed and that I should call back when they are open.  No voice mail or offer to call back the next day.

Dr. Walter Rolandi examination of NSTAR Customer Service:    

 

Patient:                                  NSTAR Power Company

Task:                                     Obtain current account balance  

Toll Free Number:              1-800-592-2000

 

The Examination Session

Just one call was required to complete the task.  The task was completed in a mere 42 seconds and the entire call lasted only 83 seconds. 

 

Call Details:

The system is a speech recognition application with an “open ended” (“How may I help you?”) Main Menu.  The user experience begins with the caller hearing:

 

Hello.  Thank you for calling NSTAR. 

For quality purposes, your call may be recorded.

I am an automated assistant.

How may I help you?

 

I responded, “I need to get my current account balance”.   After about a three second pause, the system replied:

 

Account balance and recent payments.

 

It was unclear whether the system response was intended as a confirmation or a question.   The intonation of the prompt implied that it was a question but this was far from obvious.  Confused, I said nothing.  After another 2.5 second pause, the system continued:

 

Please hold while I look up your account.

OK. 

What account number are you calling about?

 

I then stated the 11-digit account number I was to use to which the system responded:

 

OK.

Your account balance is $289.79.

 

At this point, my task was complete.  The system proceeded however with an additional 14-plus seconds of unasked for information about the account.  The additional account information was followed by a legalese disclaimer and a series of yes/no questions offering to provide even more additional account-related information or services, all of which I declined.  The system then thanked me for calling, read me a web-ad, said, “Goodbye” and hung up.

 

The Diagnosis

1.      This system appears to be very healthy indeed.

2.      Speech recognition was perfect.

3.      Task completion was quick and easy.

4.      Some system feedback was less than clear. (“OK.”)

5.      The production quality of a number of prompts is questionable.

6.      Some prompts sound garbled or electronically distorted. 

7.      The functionality of some prompts is questionable (“I am an automated assistant.”)

8.      Some prompt content could be trimmed away.

9.      Small silences could be reduced.

 

The Prescription

1.      Clean up distorted prompts and rerecord as necessary.

2.      Provide more meaningful user feedback.

3.      Eliminate the few unnecessary prompts and prompt content.

4.      Try to eliminate the pauses that follow user speech.

 

The Prognosis

Excellent.

 


 

Walt and Walter’s VUI View: Central Maine Power (CMP)

The Caller’s View from Walt:

I called CMP to obtain information on my account.  They have a structured TouchTone self-service.  Lots of long-winded instructions.  They tell you about their web site and to pay attention since their menu has changed.  I requested my account balance via the automated service.  It took over 2 minutes to accomplish this.  This was really tedious to use.  After getting my account balance, I requested to be connected to a human.  This was accomplished quickly (less than a minute).  The agent had my account information available and was quickly able to answer the question that I had.  The availability of CTI compensated for the laborious self-service.  The only benefit of self-service is to access information after hours when humans are not available.  During business hours, agents appear to be readily available and provide a much more pleasant and quicker response than the self-service.  No point in using the self-service during normal business hours. 

I attempted to talk to a human after normal business hours.  I was informed that they were closed and that I should call back when they are open.  No voice mail or offer to call back the next day.

 

CMP

Dr. Walter Rolandi examination of Central Maine Power Customer Service:   

Patient:                                  Central Maine Power Company

Task:                                     Obtain current account balance  

Toll Free Number:              1-800-750-4000

 

The Examination Session

One call was required to complete the task although another call was made to determine whether the system supported dial through (it does).  The task required 126 seconds to complete.  The entire call lasted 145 seconds. 

 

Call Details:

The system is a highly structured DTMF only application.   The user experience begins with 46 seconds of preemptive instructions, web ads and additional unrequested information.   It is too wordy to transcribe.  Eventually, the system instructs the caller:

 

If you’re calling from a touch-tone phone, please press one.

 

Upon compliance, the system continues:

 

With this automated phone system, you may use several self-service options.

For self-service options, press one.

 

Upon compliance again, the system read out the company’s variant of the IVR cliché:

 

Please listen carefully before selecting from one of the four options.

 

The first option turned out to be “check your account balance”.  After listening to the entire menu, I entered DTMF1.   I was then prompted for my account number which I keyed in.  The system then laboriously read the account number back, requiring me to confirm it with (yet another) DTMF1 response.  The confirmation process began with the pointless assertion:

 

We have recorded your 13-digit account number as….

 

At this point, I was finally read the account balance followed by some additional unrequested account information.  I was then told that additional account services were available by pressing (guess what) DTMF1.  My task complete, I hung up at this point.

 

The Diagnosis

1.   This system is sadly afflicted with many of the telltale symptoms of poor DTMF IVR design.

2.   It is tedious and laborious to use.

3.   It is full of unnecessary prompts and prompt content.

4.   It appears at times to be illogically organized.

5.   It requires what appear to be unnecessary or pointless steps.

6.   Task completion is painstakingly slow.

7.      Overall production quality is sub par.

8.      The confirmation strategy is irrational and unnecessary.

 

The Prescription

1.      Get rid of the entire 46-second preemptive preamble.

2.      Streamline the organization of the system in order to make its functionality more immediately obvious to the caller.

3.      Eliminate any dialog state that is not absolutely required.

4.      Eliminate or trim down any unnecessary prompts and prompt content.

5.      Eliminate the confirmation strategy.

6.      Adopt industry best practices for DTMF design.

 

The Prognosis

Reasonably good.

 

 

 

 

Sep 7 2008

Walt and Walter’s VUI View: USCIS

The Caller’s View from Walt:

I needed to locate the nearest office of the US Citizenship & Immigtation Service (USCIS).

USCIS is a government agency comprised of 15,000 federal employees and contractors working in approximately 250 Headquarters and field offices around the world.

 

They are responsible for the administration of immigration and naturalization adjudication functions and establishing immigration services policies and priorities.

 

I called the number that was listed on the GetHuman web site (800-375-5283).  After the press 1 for English option and being told to “please pay attention since the menu had changed, I ran into the longest main menu that I’d ever encountered.  The prompts just went on-&-on.  It used all of the keys for options and when it was done with all of the options, went on a long harangue about going to the Internet to get all of the information on the options and much more.   It even took the time to suggest that if you did not have access to the Internet in your home or business, then you can go to the local public library to obtain Internet access.  At the conclusion of the IVR session, I was asked if I wanted to take a survey.  I took it and it proceeded to ask about 10 questions on what I thought of the IVR system.  The US CIS appears to be trying very hard to provide a good deal of information to their callers and failing miserably.  They somehow think that using 10 words when it can be done quite well with a single word or two, is an effective way of communicating.  This is one of the most unnecessarily verbose implementations that I have ever run into.  It is so tedious that it is virtually impossoble to use without becoming highly irritated.

 

Dr. Walter Rolandi examination of US CIS Customer Service:     

 

Patient:                                  US Citizenship & Immigration Service

Task:                                     Obtain location of nearest US CIS office  

Toll Free Number:              1-800-375-5283

 

The Examination Session

Only one call was necessary to complete the task.  The task required 213 seconds in total or approximately three and a half minutes. 

 

Call Details:

The system is a highly structured DTMF only IVR.  Calls are answered with a 20 second greeting that includes an unnecessarily wordy and poorly thought out Spanish option.   The male voice talent is polished, professional and of fine production quality.  Selecting the English language route leads to the familiar IVR cliché, “Please listen carefully.  The options on our Main Menu have changed”.   This is then followed by what appears to be the longest, most verbose Main Menu I have ever encountered. 

 

Normally, the Call Details section includes transcription of the patient IVR dialog through the termination of the Main Menu.  In this special case, transcription would require more time than I have to devote to the examination.

 

Suffice it to say that the Main Menu includes no less than ten (10) choices, all expressed in pointlessly verbose language and many of which include preemptive prompt content.   Additionally, the Main Menu includes a very lengthy web ad.  The Main Menu ultimately offers an option to press “*99 to take a 30 second survey”.

 

Navigating to the end of the system’s Main Menu required 121 seconds (over two minutes).  After this arduous journey, I entered DTMF4 to find the nearest US CIS.   This lead to a 27 second submenu wherein the user is required to enter DTMF1 to obtain the location information that was previously made available as choice four on the Main Menu.  Entering DTMF1 lead to a request for the caller’s zip code.  Upon entering the zip code, a nine second silence followed by a readout of the requested information.  The office location information sounded like a locally recorded insert.  A female voice talent is used.

 

After the location information was readout, the “system voice” returns with some more verbose instructions about how to repeat the information, return to the Main Menu and hang up.

 

The Diagnosis

1.      The system could serve as a textbook example on what not to do when designing an IVR.

2.      Overall production quality is good.

3.      Spanish option is too long and poorly planned out.

4.      Main Menu is far too long.

5.      Prompts are far too verbose.

6.      Preemptive prompt content abounds.

7.      Unexplained silence follows zip code entry.

8.      Pace is slow.

9.      System is unbelievably tedious and time consuming to use.

 

The Prescription

1.      The VUI Surgeon’s Scalpel is desperately required

2.      Extract the Spanish option or rethink its design and length.

3.      Cut the number of menu choices.

4.      Debride all prompts of unnecessary prompt content.

5.      Excise all preemptive content.

6.      Investigate the silence between zip code entry and response.

 

The Prognosis

Grim without radical surgery.

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

Aug 7 2008

Walt and Walter’s VUI View: AARP

The Caller’s View from Walt:

I needed to obtain a replacement AARP membership card.  I called the main AARP number (888-687-2277).  After asking me if I was a member (I responded that I was) it presented a four item menu to me.  One of the menu items was Membership Card, so I said “Membership Cards”.  It then asked me for my 10-digit membership number.  I responded by saying “I don’t know it”.  It then came back and said: “Did you mean “I don’t know it”. Please say “yes” or “no””.  I said “yes” and it responded with: “please hold for the next available representative”.

The inappropriate error recovery after I said “I don’t know it”, ruined a really good call experience.  No reason exists to obtain a confirmation that I really said that I didn’t know my membership number.  Any response other than me providing the number should have caused the system to simply go ahead and transfer me to a CSR.  Jerking the caller around with a “did you mean?” is just wrong.

When I reached a CSR, she did not know what selection I had made at the menu and I had to repeat that I needed to obtain a replacement membership card.

I made 4-5 calls and each time it asked me whether I was a member or not.  The system does capture the CallerID but appears to do very little with it. 

 

Dr. Walter Rolandi examination of AARP Customer Service:       

Patient:                                  AARP Service IVR

Task:                                     Find out membership cost

Toll Free Number:              888-687-2277

 

The Examination Session

Two calls had to be made to the system before concluding that it apparently does not support the task.  There is no obvious way to determine membership costs using the system.  The two calls consumed approximately 100 seconds. 

 

Call Details:

Calls are answered with a 2 second audio-icon followed by cheerful and professional sounding female voice saying: 

 

Welcome to AARP!

This call may be recorded for quality.

After approximately a 2.5 second pause, the system states:

 

Please say yes or no.

Are you or the person you’re calling for a member of AARP?

 

Not being a member, I answered, “No” after which the system told me:

 

Please hold for the next available representative.

 

Not wishing to bother “the next available representative”, I hung up.

 

I immediately called back.  On this call, I answered, “Yes” to the initial question and heard the following: 

 

Main Menu

Please say if you’re calling for AARP products and services

Insurance

Membership cards

Or to check expiration date

 

I ventured a guess as to where I might find the cost of membership and said, “Products and services”.  After a 3.0 second silence, the “products and services” menu was read.  There was nothing on the menu that suggested membership costs although its fourth (and last) option was to say, “Something else”.  Saying, “something else” lead to another menu listing several other services, mostly having to do with travel planning.

I returned to the Main Menu and wondered about a bit more but eventually concluded that membership cost information was probably not available through the system.

 

The Diagnosis

1.      The system seems to enjoy good health.

2.      It is well designed and easy to navigate.

3.      Overall good production quality is good.

4.      Its menus are short and well land marked.

5.      Pace is slow but this is probably by design, given that the system primarily serves the aged.

6.      Some silences are probably too long

7.      Access to a human is never far away.

 

The Prescription

1.      Investigate the silences between prompts and trim them to a universal length

2.      Play audio-icon during (not before) the welcome prompt

 

The Prognosis

Excellent.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

July 9 2008

Walt and Walter’s VUI View: The Internal Revenue Service (IRS)

The Caller’s View from Walt:

I needed a Form 8109-C for my quarterly tax deposits. I called the main IRS number (800-829-1040).  After a long introduction, I finally got a menu of four choices: 1st was to obtain information on the rebate.  None of the other choices mentioned obtaining tax forms, so I selected an option that offered to provide information on filing my taxes.  After that, the 1st menu choice was: “To order tax forms or publications, press 1”  Thinking that I had found it, I immediately pressed 1. It responded with: “If you know the the form or publication that you need, press 1, if you don’t the form or publication, press 2.”  I responded with a 1.   The system then responded with a long monologue about the status of the rebate payment.  To confuse things, this time they referred to it as the “stimulus payment”.  At this point I bailed out of the IRS IVR and (after a 10 minute wait) was connected to a human that took my order for the form that needed.

 

Not being able to order a form via the IRS IVR is awful.  It is a waste of my time as well as the time of the agent.  It appears as though they recently added the rebate information into the IVR and this somehow managed to mess up the form request service.  Beyond this, the IRS self-service is just very poorly done.

 

Dr. Walter Rolandi examination of Internal Revenue Service (IRS):        

Patient:                                  Internal Revenue Service IVR

Task:                                     Order a 1040 form

Toll Free Number:              800-829-1040

 

The Examination Session

Three calls were made to the system before I abandoned the task.  There is no obvious way to order forms using the system as I found it on the examination date.  The three calls consumed approximately six (6) minutes to complete. 

 

Call Details:

 

A Main Menu (of sorts) appears after approximately 26 seconds of the above preliminaries.  It begins:

 

If you are calling for information about the tax rebate, press 7.

For recorded tax topics, press 1.

For questions about your refund, press 2.

For all other automated services press three.

To hear these options again, press 9.

 

Thinking “all other automated services” was the logical choice, I entered DTMF 3.  This lead to several DTMF beeps (apparently a line transfer), five seconds of silence and an instruction to enter either a social security or employer number in order “to access your account information”.  Confused, I did nothing.  Approximately six seconds later, the system repeated the request.  Failing to see what my social security number had to do with ordering a 1040 form, I hung up.

 

I immediately called back, again experiencing everything above but paying special attention to the Main Menu, assuming that I had reacted to it incorrectly on the initial call.  Again I heard:

 

If you are calling for information about the tax rebate, press 7.

For recorded tax topics, press 1.

For questions about your refund, press 2.

For all other automated services press three.

To hear these options again, press 9.

 

Still confused but wanting to be sure, I entered DTMF 9 to hear the options again.  Astonishingly, the options were not repeated.  Female #1 simply restated the greeting, normal business offices and the web ad followed by Female #2 doing the same but in Spanish.  The system then promptly hung up.

 

Again, I immediately called back and navigated through to the Main Menu.  This time I entered DTMF 1 to see if “recorded tax topics” could lead to information about ordering forms.   This lead to another (apparent) transfer, a number of preemptive prompts and, eventually, an announcement that “there are seventeen (17) main categories for which there are topics”.   After listening to the first seven topics, I hung up. 

Note that it is possible that the system provides information about form ordering somewhere between topic eight and topic seventeen.

 

The Diagnosis

1.      System is deathly ill.

2.      It exhibits repeated violations of the de facto DTMF standard.

3.      Overall organization is illogical.

4.      Jargon is present.

5.      Spanish option is present.

6.      Web ads are present.

7.      Main Menu is illogically laid out.

8.      One submenu had 17 (!) items.

9.      DTMF entry is awkwardly (if not stupidly) used of on 17-item submenu.

10.  Pointless preemptive prompts are present.

11.  Pace occasionally seemed slow.

12.  Overall good production quality is wasted on the senselessness of the system’s organization and function.

 

The Prescription

1.      Completely reorganize the entire system.

2.      Adopt de facto DTMF standard.

3.      Eliminate jargon.

4.      Eliminate Spanish option.

5.      Eliminate web ads.

6.      Eliminate preemptive prompting.

7.      Speed up overall interactive pace.

 

The Prognosis

Without the most drastic intervention, very poor indeed.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

 

 

June 9 2008

Walt and Walter’s VUI View: The Annual Credit Report Service

 

The Caller’s View from Walt:

An amendment to the federal Fair Credit Reporting Act requires each of the nationwide consumer reporting companies – Equifax, Experian, and TransUnion – to provide consumers with a free copy of their credit report, at their request, once every 12 months.  I called the Annual Credit Report Service to order a credit report.  This service is provided by the three (3) major consumer credit reporting companies.  I had previously ordered a credit report from this service, so I was a repeat caller.  You can order three (one from each of the credit reporting companies) reports each year, so most calls are repeat calls, since it makes sense to not obtain the three reports at the same time.  The service is oblivious to this and always assumes that you are calling for the first time.  It forces you to listen to an excessively long and unnecessary monolog.  Overstrike is ignored.  The voice relentless continues to babble on describing the service and what it does.  It describes in detail all of the ways in which you can order a credit report such as the Internet and mail.  The system somehow thinks that the consumer that used the telephone to request a fee credit report, needs to be told over-&-over again what the service is, how it works and the alternate ways that they can request a credit report. Despite using your phone number to identify who you are and what your address is, historical information is not used.  With your phone number, it would be easy to determine that you are a repeat user and that the excessively long monolog can be by-passed.   They do not provide any human backup so the only option is to just suffer through the horribly implemented self-service.  The intent of the awful implementation appears to be to discourage consumers from requesting a credit report.

 

Dr. Walter Rolandi examination of Annual Credit Report Service:                                   Annual Credit Report Service

Task:                                     Order a report

Toll Free Number:              877-322-8228

 

The Examination Session

Only one call was necessary to complete the task.  The process took just over six (6) minutes however.  It was at times, laborious, painstaking and annoying.

 

Call Details:

Normally, specifics of the interactive dialog are included in this section.  However, given the length of the call, no effort to transcribe its entirety was made.  Some specific pieces of the dialog are reviewed.

The system answers and the caller hears an eight-plus second initial greeting spoken by a professional sounding female voice talent.  It then launches into a 24 second monologue describing what the system is and what it exists to do. 

 

Eventually, the system instructs callers calling from their home phones to “… press or say, one.  If not, press or say, two”.  This is the first incidence of an unnecessarily wordy method for negotiating either-or logic branches within the application.   Variants of the template unfortunately are unremittingly repeated throughout the interaction.

 

Upon responding, the caller is treated to a 15 second “web ad” followed by a five second prompt offering to provide instructions on how one might submit their credit report request in writing.   At a point soon thereafter, one finally arrives at the dialog state where the process of actually making a request begins.  In my case, navigating to this state had required 99 seconds.

 

After an additional four minutes and 25 seconds, I successfully completed the credit report order.  The process included some notable speech recognition failures.  Among them were one failure to recognize the utterance, “no” in response to a “yes/no” question and two failed attempts to accurately recognize the spelling of my name.  

 

Overall, the interaction was riddled with lengthy if not actually unnecessary prompts.   Many were preemptively instructive and many of the preemptive prompts were needlessly repeated throughout the session. 

 

Some user inputs seem to be confirmed unnecessarily.  The system has a pointlessly verbose and questionably effective error recovery strategy and, at times, the system prompts appear to suggest that it has conversational abilities that it clearly does not possess.

 

The Diagnosis

1.      This system suffers from neuro-linguistic infirmities.

2.      It seems to take every opportunity to say something that seldom if ever needs to be said.

3.      It also seems to say things that could be easily said with just a few words in far too many words.

4.      There are many preemptive prompts.

5.      Its time-to-task-completion is painfully slow.

6.      Overall production quality is good.

7.      Speech recognition should probably be tuned.

8.      Some prompts are artificially conversational.

9.      System response times are occasionally slow.

 

The system’s health could be dramatically improved with some effective neuro-linguistic programming.

 

The Prescription

1.      Excise all unnecessary prompts and prompt content.

2.      Eliminate preemptive prompting.

3.      Streamline the time-to-task-completion.

4.      Investigate speech recognition accuracy issues and tune accordingly.

5.      Tone down the suggestion of “conversational” abilities.

6.      Investigate response time issue.

 

The Prognosis

Very good.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

 

May 9 2008

Walt and Walter’s VUI View: The American Airlines Customer Support Line

The Caller’s View from Walt:

I called American Airlines to locate a flight from Boston to New York.  I requested “Flight and Gate information” from a four item menu.  Even so, it came back and confirmed that I had requested Flight and Gate Information.  I finally nailed down the flight that I wanted with it repeating everything that I said and asking for confirmation of each item.  I then attempted to get to a CSR to book the flight.  Waited in a queue for about five minutes.  It did not tell me the wait time or offer to call me back.  When I finally did get to a CSR, she did not have the information that I had provided to the flight information system and I had to provide it to her all over again.

American Airlines makes using the self-service an unpleasant experience. It has long and often unnecessary prompts and parrots virtually everything back to you.  When you get to the CSR, you need to repeat everything to the CSR again since nothing is passed from the self-service to the CSR.

 

Dr. Walter Rolandi examination of American Airlines customer support line

Patient:                                  American Airlines

Task:                                     Find out if service available from one city to another.

Toll Free Number:              800-433-7300

 

The Examination Session

The call was answered by a directed dialog speech recognition system.  The call was apparently transferred to another speech recognition system and eventually back to the original system that appeared to answer the call upon saying, “Main Menu”.  Only one call was required to complete the task although it did entail two “loops” through the system.  

 

Call Details:

The system answers the call with a pleasant female voice talent saying:

Thanks for calling American Airlines.  Para Español, diga, “Español”.

These prompts are immediately followed by:

Main Menu.

Please say reservations, flight and gate information, Advantage services or more options.

Upon my saying, “Reservations”, the system cheerfully responded:

Ok. Reservations.  If that’s not right, say, “Go back.”

After a pause of almost two seconds, the system continues:

Now which one would you like? 

Fares, new reservation, existing reservation or flight and gate information.

After clearing responding, “Flight and gate information”, the system replied:

I think you want, “Flight and gate information”.  Is that right?

Answering, “Yes”, the system continued with:

The next time, you can get here more quickly by saying, “Flight and gate information” at the Main Menu.

This call may be recorded.

Next came three seconds of silence followed by four seconds of music-on-hold and an apparent call transfer to another system:

Welcome to the flight information system for American Airlines, American Eagle, American Connection and other American marketed code share flights.

The intonation and pace of this prompt were unnatural, suggesting that prompt content had been electronically edited to include or delete some information.  The system continues:

To get you up to date on departure and arrival information, I’m going to ask you a few questions.   First, what’s the flight number?

A pause of .82 seconds follows whereupon the system announces:

It’s Ok to say, “I don’t know”.

I said, “I don’t know”, the system went silent for approximately two seconds and then carried on:

Ok let’s find out what flight you want.  What’s the departure city?

“Boston”, said I.

Next!  Tell me the arrival city?

“Chicago”.

Thanks!  Now is that for arrival or departure info?

“Departure”.

And finally, around what time does that flight depart?

“Noon”.

Lemme confirm that….

A laborious restatement of all this information subsequently follows after which the system located a relevant flight.   The entire process required two minutes and twenty-three seconds to complete.

 

The Diagnosis

1.      This system is reasonably healthy but it is not without some infirmities.

2.      It suffers from slight production quality issues.

3.      Volume, intonation and pace vary significantly from prompt to prompt.

4.      At times, pace is so fast that it makes comprehension difficult. 

5.      Interaction is artificially conversational.

6.      At least one pause invites false starts (i.e., “First, what’s the flight number?” (.82 second pause)  “It’s Ok to say, “I don’t know””.

7.      Persona borders on the annoying.

8.      Many prompts are unnecessarily long.

9.      Several prompts contain useless content (i.e., “To get you up to date on departure and arrival information…”)

10.  There are some preemptive prompts and “hints” present.

11.  Prompts contain jargon (i.e., “American marketed code share flights”)

12.  Some IVR Clichés are present.

13.  System response times seem inconsistent.

The system’s health could be greatly improved with some minor surgery.

 

The Prescription

1.      Clean up production quality issues with volume, intonation and pace.

2.      Tone down the animation in the persona.

3.      Re-examine pause and timeout practices.

4.      Speed up interaction.

5.      Eliminate IVR Clichés, jargon, preemptive prompts and “hints”.

6.      Trim all prompts of unnecessary content.

7.      Investigate response time issue.

 

The Prognosis

Quite good.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

Apr 12 2008

Walt and Walter’s VUI View: The Tufts Healthcare Customer Support Line

The Caller’s View from Walt:

I needed to obtain a 1099HC from my health insurance company to prove to the state of Massachusetts that I had health insurance.  I called the Tufts Healthcare customer support line.  I was greeted by a voice that sounded identical to the Amtrak Julie persona.  So much for unique personas.  I was calling at 5:30 in the evening.  This turned out to be outside of normal business hours.  It dumped a menu at me that gave me the choice of 1) either checking my bill; 2) obtaining information re my pharmaceutical coverage; or 3) or filing an appeal of a denial of services.  After the menu, which was way to verbose, it then announced that they were closed and that if I wanted anything else, I should call back when they were open for business.  Imagine that. Not even voice mail.

I called back the next day.  Got a 6 item menu for identifying the type of caller that I was.  This could have been eliminated by looking at my CallerID.  Next was a menu that had claimed to have 7 choices and then gave me 8 choices.  What I was calling about was not one of the choices, so I chose the “none-of-the-above”  choice.  When I finally reached a CSR, none of this information was passed which meant that I had to provide it all over again.   By not having CTI implemented, Tufts Healthcare is wasting my time as well the time of their CSR.  

 

Dr. Walter Rolandi examination of Tufts Healthcare customer support line

Patient:                                  Tufts Healthcare

Task:                                     Obtain name, address and phone number of an eye care provider

Toll Free Number:              800-701-9000

The Examination Session

Initial system branches off to at least one other system.   First system is a “press or say” directed dialog IVR.  Second system supports slightly broader speech recognition but also a directed dialog.   System had to be called four times in order to discover how to complete the task.  

Call Details:

The system answers the call with a female voice talent saying:

Thank you for calling Tufts Health Plan Medicare Preferred.  For quality purposes, calls may be monitored and recorded.  Please listen carefully to the following options as the menu has changed.

After this greeting and IVR cliché ordeal is complete, (almost 11 seconds), one hears this monstrous 30-plus second Main Menu:

            If you are a provider or are calling on behalf of a provider, press or say one.

 

If you are a member of our HMO plan, press or say two.

 

If you are a member of our PPO plan, press or say three.

 

If you are a member of our Private Fees for Service Plan, press or say four.

 

If you are unsure of your plan type, please look in the upper right hand corner of your ID card.

 

If you’re interested in becoming a member of Tufts Health Plan Medicare Preferred, press or say five.

The system then pauses a mere 1.5 seconds before announcing:

Please make your selection now.

I had been prepared to say, “Two” since I was calling as an HMO member but the “Please make you selection now.” prompt confused me and I paused.  After approximately 3.0 seconds, the system continued:

If you’re calling from a rotary phone, please stay on the line. To repeat this menu, say repeat or press star.  To end this call, simply hang up.

Almost an entire minute (59 seconds) has now passed since the system answered the call.   It finally permitted me to say, “Two”.  After another 2.0 seconds and without any additional feedback, the system launched into a 63 (!) second submenu which began:

Please listen carefully and select one of our seven options….

This prompt was also punctuated with the utterly useless statement:

Please make your selection now.

It was not immediately clear which menu choice I should indicate in order to locate an eye care provider.   One erroneous choice was guessed after which I hung up.  I then called back two more times, indicating option “three” because it said something about “Vision Care” (?).  Both times the system responded that my call was being transferred to “Eye Med Member Services” (?) and, assuming that a human would soon answer the call, I hung up.  On the fourth and last call, I did not hang up.  After approximately 5 seconds, a line was heard ringing.  The ringing lasted for over 9 seconds whereupon another system appeared to answer:

Welcome to Eye Med Vision Care (?).

There are several things that I can help you with.  Please say one of the following:

Provider locator, plan benefits, claim forms or claim status.

Upon saying, “Provider locator”, the system then read an 11 second prompt describing my vision plan and requesting a zip code where to look for providers.  I spoke the zip code, “01720” after which the system said,

Hold on while I look that up.

The system soon came back saying that it hadn’t been able to locate any provider within 100 miles of the zip code I that I had provided.  What followed was a series of nonsensical instructions that required me to return to the Main Menu in order to say another zip code.  I eventually muddled through, finally completing the task in just over 5 minutes and 15 seconds!

The Diagnosis

1.      This system is deathly ill.

2.      The system (or systems) supports inconsistent interactive modalities.

3.      System attempts to automate far too many tasks.

4.      Prompts are unbelievably long.

5.      Preemptive prompts are abundant.

6.      Menus have too many options and are far too wordy.

7.      The system is painstakingly slow to use.

8.      Feedback is often insufficient.

9.      Logical organization of the system is occasionally unapparent.

10.  Timeout timers are often inappropriate.

11.  Speech recognition accuracy seems sub par.

12.  A bevy of IVR Clichés is present.

13.  System response to input seems slow.

14.  Production quality is low-to-medium.

Performing this seemingly simple task using this system (or systems) revealed the severity of its ailments.   Only the most profound interventions can save this patient.         

The Prescription

1.      Bust up the various components into smaller, more manageable and functionally rational systems.

2.      Adopt a radically different design philosophy.

3.      Employ a consistent interactive modality.

4.      Provide meaningful feedback.

5.      Speed up interaction.

6.      Improve ASR accuracy.

7.      Eliminate IVR Clichés.

8.      Shorten menus.

9.      Trim all prompts.

10.  Improve production quality.

 

The Prognosis

Without fundamental and radical surgery, I am afraid there is little hope.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

Mar 7 2008

Walt and Walter’s VUI View: The Charter Communications Customer Support Line

 

The Caller’s View from Walt:

 

I attempted to find out what the cost and availability of the Charter Communications Triple-play package was by calling the telephone customer support number (866-472-2200). After a brief greeting, and the press 1 for English prompt, it asks you to enter your phone number, which I promptly did.  Charter Comm’s appears to ask you to provide your phone number for no purpose other than to irritate and waste the time of the caller.  They do nothing with the number.  The menu offers me four options.  They assume that I’m already a subscriber since they are about billing, scheduling a technician appointment or obtaining technical assistance.   They had forced me to provide my phone number from which they should have easily been able to determine whether or not I was a subscriber.    I selected the 4th menu item which was for purchasing services from Charter Communications.  The options were a new subscription and a pay-per-view movie.  Again, offering me the pay-per-view option was inappropriate for them to do for a non-subscriber.   I was next connected to a CSR.  I asked him if he had my phone number or knew what I had requested.  He said that he did not.   He also indicated that he handled all types of calls from billing to new services.

 

These people want to be your telephone company.  Yet, they don’t support rather rudimentary telephony functions such as CallerID or CTI.   Why they bother to waste the caller’s time listening to a menu and responding to it, totally escapes me.   You are routed to the same CSR no matter what you input, so the entire IVR process serves no purpose other than to waste the caller’s time.

 

Dr. Walter Rolandi examination of Charter Communications customer support line