Home
   
e-ASRNews Log-on
     
Subscription Info
 
Market Reports 
 
Speech Industry Links
 
Examples of Good VUIs
   
Repairs for wrecks
   
Get2human Consulting Services
  
Tern Systems
 
Get2Human
 
Speech Industry Job Opportunities

Repairs for Wrecks 
NYC Trip Planner  Milwaukee Journal Sentinel  Commonwealth Choice   NSTAR    CMP    US CIS  AARP  IRS  The Annual Credit Report Service  American Airlines   Tufts   Charter   Washington Post   NY Times   WSJ   SSA   Dell   Virgin Mobile    VerizonBroadband   Cingular   Apple  Federal Trade Commission  Eureka   FedEx  Comcast    United Airlines   Verizon    CareMark   Amtrak   USPS   aol   Bank of America  

   
Telephone self-service is able to provide real value to consumers. It makes access to information available 24/7. It is available without waiting in line. Consumers can purchase things. They can check availability. They can check the status of an order. They can obtain directions. In addition to making customers happy, telephone self-service is also able to save money for the enterprise that is providing it.

Unfortunately, this situation is rarely achieved. Telephone self-service is intensely disliked by many consumers because it is implemented very badly, especially by companies who cut corners just for the sake of extra savings. The caller wants a quick & easy experience and most telephone self-service instead provides an experience that is slow & difficult.

We present here a few of these telephone self-service implementations that were evaluated during the last months. These were selected mostly at random and it's unlikely that they represent the worst implementations.

Table 1 summarizes the results of this testing. We have adapted the www.get2human.com Voice User Interface (VUI) Standard as the basis for our evaluation. The scoring/rating scheme is simple to understand and scoring of a particular VUI is relatively easy to do. All that is needed is to make a phone call and observe how the system responds. The scheme that we are using has some obvious limitations. The fact that we give each item equal weight is certainly arguable. On the other hand, despite its simplicity, it appears to yield results that seem to correlate with the relative goodness of the self-service system.

For each of the VUIs that we tested, we provide a discussion of it from the perspective of the caller. This is done by Walt Tetschner, who has suffered through many poorly implemented telephone self-service implementations. We also provide a clinical analysis of the system from the viewpoint of a highly-regarded VUI designer (Dr. Walter Rolandi). This identifies what the system is doing incorrectly and offers recommendations regarding what should be done to repair it.

Table 1

Get2Human Telephone Self-service Rating
Get2Human item NYC Trip Planner Milwaukee Journal Sentinel Common-wealth Choice Dell SSA WSJ NY Times Washington Post Charter Commun-ication Tufts Health
care
American Airlines The Annual Credit Report Service IRS AARP US CIS NSTAR Central Maine Power (CMP)
The caller must always be able to dial 0 or to say "operator" to queue for a human. 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 1
An accurate estimated wait-time, based on call traffic statistics at the time of the call, should always be given when the caller arrives in the queue.  A revised update should be provided periodically during hold time. 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 NA NA
Callers should never be asked to repeat any information (name, full account number, description of issue, etc.) provided to a human or an automated system during a call. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
Callers should always be offered the option to be called back when a human is not available. This option should be offered periodically during hold time. If 24 hour service is not available, the caller should be able to leave a message and be called back promptly when humans are available.. 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0
Speech applications should provide touch tone (DTMF) fall-back, where appropriate.  0 1 NA 0 0 0 NA 1 NA 1 1 1 NA 1 NA 1 NA
Callers should not be forced to listen to long/verbose prompts.  0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0
Callers should be able to interrupt prompts (via dial-through for in DTMF applications and/or via barge-in for speech applications) whenever doing so will enable the user to complete his task more efficiently. 0 0 0 0 0 0 1 1 0 0 1 0 0 1 0 1 1
Do not disconnect for user errors, including when there are no perceived key presses (as the caller might be on a rotary phone); instead queue for a human operator and/or offer the choice for call-back.  0 1 0 1 0 0 0 1 1 0 1 0 0 1 0 1 1
Default language should be based on consumer demographics for each organization.  Primary language should be assumed with the option for the caller to change language.   (i.e. English should generally be assumed for the US, with a specified key for Spanish.) 1 1 0 1 0 1 1 1 0 1 1 1 0 1 0 1 1
All operators/representatives of the organization should be able to communicate clearly with the caller  0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1
Gethuman Subtotal (Average score) .10 0.60 0.11 0.20 0.10 0.20 0.33 0.70 0.33 .30 0.6 .30 0.11 0.90 0.11 0.78 0.75
Recognize me by caller ID (home or mobile) and then let me enter a four digit PIN to authenticate, so you then have my info on file and don't make me type in long account numbers etc. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
No "Your call is important to us" greeting.  1 0 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1
No "You can access our website to answer most questions" message. I already know that. 0 1 0 0 0 0 1 1 1 1 0 0 0 1 0 1 0
No "Please listen carefully, since our menu has changed" message.  1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 1 0
Do not play advertisements to me 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Do not speak to me in legalese or corporate language 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 1
Is it a Quick-&-Easy experience? -3 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0
Understand me when I speak a natural number 0 1 NA 0 0 0 NA NA NA NA NA NA 0 NA NA 0 NA
Total Score (Average) 0.00 0.61 0.31 0.33 0.28 0.28 0.50 0.65 0.50 0.35 0.59 0.41 0.24 0.88 0.19 0.76 0.60
 
Get2Human Telephone Self-service Rating (continued)
gethuman item BofA AOL USPS Amtrak CareMark Verizon United Airlines Comcast Eureka FedEx FTC Apple Cingular Verizon Broadband Virgin Mobile
The caller must always be able to dial 0 or to say "operator" to queue for a human. 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
An accurate estimated wait-time, based on call traffic statistics at the time of the call, should always be given when the caller arrives in the queue.  A revised update should be provided periodically during hold time. 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0
Callers should never be asked to repeat any information (name, full account number, description of issue, etc.) provided to a human or an automated system during a call. 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
Callers should always be offered the option to be called back when a human is not available. This option should be offered periodically during hold time. If 24 hour service is not available, the caller should be able to leave a message and be called back promptly when humans are available.. 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0
Speech applications should provide touch tone (DTMF) fall-back, where appropriate.  1 1 1 1 1 0 1 1 0 1 0 0 NA 1 1
Callers should not be forced to listen to long/verbose prompts.  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Callers should be able to interrupt prompts (via dial-through for in DTMF applications and/or via barge-in for speech applications) whenever doing so will enable the user to complete his task more efficiently. 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0
Do not disconnect for user errors, including when there are no perceived key presses (as the caller might be on a rotary phone); instead queue for a human operator and/or offer the choice for call-back.  1 0 0 0 1 0 1 1 0 1 0 0 0 1 0
Default language should be based on consumer demographics for each organization.  Primary language should be assumed with the option for the caller to change language.   (i.e. English should generally be assumed for the US, with a specified key for Spanish.) 1 0 1 1 0 0 0 0 1 1 0 1 0 1 0
All operators/representatives of the organization should be able to communicate clearly with the caller  1 0 1 1 1 1 0 1 1 1 1 1 1 0 0
Gethuman Subtotal (Average score) 0.50 0.10 0.40 0.40 0.30 0.10 0.20 0.40 0.20 0.60 0.10 0.30 0.11 0.30 0.10
Recognize me by caller ID (home or mobile) and then let me enter a four digit PIN to authenticate, so you then have my info on file and don't make me type in long account numbers etc. 0 0 0 0 0 0 0 1 0 1 0 0 0 0 NA
No "Your call is important to us" greeting.  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
No "You can access our website to answer most questions" message. I already know that. 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1
No "Please listen carefully, since our menu has changed" message.  1 1 1 1 0 1 1 1 1 1 0 0 1 1 1
Do not play advertisements to me 0 0 1 1 1 0 1 1 1 1 1 0 0 0 1
Do not speak to me in legalese or corporate language 0 0 1 1 0 1 1 1 1 1 0 1 0 0 0
Is it a Quick-&-Easy experience? 0 0 1 0 0 0 0 1 -7 1 0 0 0 0 0
Understand me when I speak a natural number 0 1 0 0 0 1 1 NA 0 1 1 NA 0 1 0
Total Score (Average) 0.39 0.22 0.56 0.44 0.33 0.28 0.39 0.59 0.00 0.72 0.22 0.29 0.18 0.33 0.29

6/11/09

Walt and Walt’s VUI View: NYC Transit Trip Planner

The Caller’s View from Walt:

I called the NYC Transit Trip Planner Voice to see how well it worked.  I had seen a lot of recent hype re the new speech-enabled planning that Nuance and Aspect Software had implemented.

I called 718-330-1234.  After a welcome greeting that was too long, I encountered a menu that used a different voice for each menu item.  It had long pauses after each item.  It only had 5 menu items, but still managed to use 1 two times.  The response to my entries was inconsistent.  Sometimes it ignored the actual input (no overstrike).  Other-times it provided the information even though it wasn’t requested. (it seemed to insist on telling the caller about the re-routing of buses on Broadway, independent of the key that I pressed).  Getting to the voice service was problematical.  Pressing 9 to “Continue” was a menu option.   This seemed to get me to the Trip Planner Voice, but you had to patiently listened to all of the menu items (over a minute).    

The speech system was a highly-directed implementation.  It explained in much detail how I could specify my departure and destination points, times, and modes of tramsportation.  Huge delays between a caller input and a subsequent response.  I was calling from a quiet office environment and it had lots of speech recognition problems.  Classical error recovery torture.  It asked me when I wanted to leave and I said “right away”.   It had difficulty recognizing this and forced me to specify a precise time.   Unnecessary confirmation of my responses.    It finally read the travel instructions to me.  It had part of each phrase spoken by the same voice and used TTS for only the variable portion.  This would have worked better if they used the TTS voice for the entire phrase, since the switching of voices is a distraction.

I indicated that I wanted to travel from The Empire State Building in Manhattan to The Yankee Stadium in The Bronx.  It was 10:30AM when I called.  The system indicated that the next subway train would not leave until 11:43AM.   This seemed strange – an hour-&-a-half wait on a busy weekday morning.  I decided to talk to an agent.  The automated system did not tell me how long the wait would be but did state that an agent would be with me in “just a moment”.  After waiting for 3-4 minutes and being told to go to the MTA website a number of times, I finally reached an agent.  No functioning CTI was in place and I had to repeat everything that I had entered into the IVR.  The agent indicated that the departure time for the train that I was looking for at 10:30AM, was 10:43AM and that the 11:43AM time that it gave me was incorrect.  I went to the NYC Transit web site www.tripplanner.mta.info   It confirmed that the departure time was 10:43AM

This is one of the very worst self-service systems that I’ve run into.  Virtually everything about the implementation seems to be wrong.  In addition to it being an irritating and time-wasting experience, the information that it provided was wrong.

 

“Dr.” Walt Tetschner examination of NYC Transit Trip Planner:

Patient:                       NYC Transit Trip Planner

Task:                                      Get from The Empire State Building to The Yankee Stadium

Toll Free Number:    1-718-330-1234

 

The Examination Session

Several calls were made to the IVR on a busy weekday morning.   Only one call was actually necessary to complete the task. Call duration was 298 seconds or approximately four and a half minutes.

Call Details:

The “system” appears to be two or more systems linked together.  The call was initially answered by a DTMF system.  A menu was then presented that used a different voice for each menu item.  Each menu item was followed by a long pause (approximately 5-6 seconds), which seemed to be giving the caller adequate time to respond.  If the caller did not respond with a key depression within this pause period, the system appeared to assume that this item was rejected and proceded to the next menu item with a different voice. 

Upon pressing DTMF9 at the last item of the “menu”, the call is apparently transferred to another system where yet another voice talent appears.  At this point, the “system” switches over to speech recognition.  A significant delay occurs to get switched to the speech system.

After a long comment about wheelchair accessibility, it finally says:

If you need to know how to get from one location to another, say plan a route.”

 I said: “Plan a route.”

 “All right. In order to plan a route, I need to know what type of starting point, your destination and when you will be traveling.”

Which borough are you starting in?”

I said: “Manhattan.”

 “Sorry I didn’t get that. Are you starting in Manhattan, Brooklyn, Queens, the Bronx, Staten Island?”

 I said: “Manhattan.”

 “I need to know which type of starting point you will be using. So, would you like to tell me an intersection, landmark, or point of interest, subway stop, or address?”

I used the landmark approach going from The Empire State Building to Yankee Stadium.  This was a tedious experience with long prompts, long pauses, and frequent speech recognition errors, where I had to repeat the input.

After it provided the travel information to me, it informed me that I could say “goodbye” and hangup to end the call.

The Diagnosis

1.      The “menu” that sequentially reads each item and waits for a response, is a grossly inefficient way to communicate with a caller. 

2.      The “system” mixes exclusive interactive modalities (DTMF only and speech only).

3.      The database access is providing incorrect information.

4.      System uses different voice talents for different phrases or portions of a phrase.

5.      Long pauses exist everywhere.

6.      A long web ad is played.

7.      Many prompts are unnecessary or far too long.

8.      The speech system seemed to have difficulties recognizing simple, easy to recognize utterances.

9.      The system stupidly apologizes to the user when it fails to “understand”.

The Prescription

1.      The existing “menu” structure should be eliminated in favor of a true menu.  Better yet, a NL implementation with agent-assisted ASR (AAA) would yield a superior solution.

2.      Adopt a single interactive modality universally supporting both DTMF and speech.

3.      Fix the database access to assure that the data that is being delivered is accurate.

4.      Add CTI and make sure that the agents are using it.

5.      Add a queue wait time announcement.

6.      Implement a consistent overstrike.

7.      Either shorten or eliminate the web ad.

8.      Use a single professional voice talent for all prompts.

9.      Eliminate or trim any unnecessary prompts and prompt content.

10.  Eliminate apologies (users don’t care when computers are “sorry”).

11.  Investigate and tune system grammars.

The Prognosis

Good if the prescription above is followed.  The problems are so severe though, that this requires virtually a total re-design of the existing system.

 

 

 

Jan 8 2008

Walt and Walter’s VUI View: Milwaukee Journal Sentinel

The Caller’s View from Walt:

I called the Milwaukee Journal Sentinel to obtain information about starting a subscription. I was greeted by the typical up-front verbage that wasted my time by forcing me to listen to bunch of unnecessary stuff like instructing me to use the MJS website.  Finally gave me a TouchTone menu, but managed to change voices to cause confusion.  I selected something that was vaguely close to what I was calling about which put me into another system that had a different voice and required speech input.  I responded to the “Something else” category by saying “new subscription”.  When requested, I entered my phone number and zip code and it came back and gave me the rate for a 1-week subscription.   The rate was $18.42 a week.  It then started to verify my name and address by spelling everything out.  The verification process that they use is painful.  Virtually everything that you say gets a repeated back to you followed by: “is that correct?  Please say yes or no?  Half of the time it does not recognize the yes response properly and you have to repeat it.  I wanted rates other than the weekly rate which seemed rather high.  The automated system didn’t appear to have this information available so I attempted to get to an agent.  After hitting 0 a few times, I was connected into a wait queue that informed me that the weather was bad and negatively impacting newspaper deliveries and that 19 people were ahead of me in the line.  It offered to call me back.  I opted for this.  After a few days, I have still not received a call-back.  I attempted to use the system again.  I entered all of my information again and then waited patiently in line.  When I finally got to the agent, she did not have any of the information that I had laboriously entered into the automated system. 

Dr. Walter Rolandi examination of Milwaukee Journal Sentinel Customer Service:   

Patient:                       Milwaukee Journal Sentinel

Task:                           Determine the cost of a new subscription.

Toll Free Number:    1-800-759-6397

 

The Examination Session

Several calls were made to the IVR over a holiday weekend.   Only one call was actually necessary to complete the task, which, if unnecessarily time consuming, was reasonably easy to complete.  Call duration was 147 seconds or approximately two and a half minutes.

 

Call Details:

The “system” appears to be two or more systems linked together.  The call was initially answered by a DTMF system.  After reading a 30-second series of introductory messages which included a web ad, the user is instructed to “press one for our automated phone system”.  These various messages were apparently recorded using three different female voices, one decidedly more professional sounding than the others.

 

Upon pressing DTMF1, the call is apparently transferred to another system where yet another voice talent appears.  At this point, the “system” switches over to speech recognition.

 

After an approximately 10-second banner prompt explaining that inclement weather had caused various delivery problems, the system says:

 

If that’s not why you’re calling, which of these can I help you with?

A delivery issue

Vacation hold

Your account or

Something else.

 

The system then immediately repeated these choices.  While the “Your account” option seemed like a possibility, I choose the “Something else” route.  This quickly led to a four-item submenu, the second choice of which was “Start a new subscription”.

 

After saying, “Start a new subscription”, the system prompted me for my telephone number.   It failed to recognize the number on the first attempt.  After the second attempt, the system read my number back correctly saying:

 

Is that right?

Just say “yes” or “no”.

 

The system then prompted me for my zip code, again asking me to confirm what it “heard” with:

 

Is that right?

Just say “yes” or “no”.

 

Oddly, it failed to recognize me when I answered, “Yes” but got it on the re-prompt.  About ten seconds of information about a payment plan was then read out, finally followed by the subscription cost amount.  Upon hearing the rate, I hung up.

 

The Diagnosis

1.      The “system” mixes exclusive interactive modalities (DTMF only and speech only).

2.      System uses two, three or four different voice talents.

3.      The web ad is an IVR cliché.

4.      Many prompts are unnecessary or far too long.

5.      The system stupidly and repeatedly instructs the user to answer its yes/no questions with, “Just say ‘yes’ or ‘no’”.

6.      While the production quality of the speech-only component is very good, the overall production quality not very professional.

7.      The speech system seemed to have difficulties recognizing simple, easy to recognize utterances.

8.      The system stupidly apologizes to the user when it fails to “understand”.

 

The Prescription

1.      Adopt a single interactive modality universally supporting both DTMF and speech.

2.      Either shorten or eliminate the wed ad.

3.      Use a single professional voice talent for all prompts.

4.      Eliminate the pointless “Just say ‘yes’ or ‘no’” prompt.

5.      Eliminate or trim any unnecessary prompts and prompt content.

6.      Eliminate apologies (users don’t care when computers are “sorry”).

7.      Investigate and tune system grammars.

 

The Prognosis

Really quite good.

 

 

 

 

 

Dec 8 2008

Walt and Walter’s VUI View: Commonwealth Choice

The Caller’s View from Walt:

Commonwealth Choice is the government coordinator for the universal healthcare program that the state of Massachusetts offers to their residents.  I called their customer support line (877-623-6765) to find out what the basic eligibility requirements were. 

Commonwealth Choice starts by playing an ad and then telling you to go to their web site, describes what it contains and what the address is.  It then instructs the caller to press 1 for English.  It then forces the caller to listen to a long menu of choices.  If you press 5 at the 1st menu, you are then requested to indicate the number of adults and children in your family and then checks that your income is below the cut-off.  If you qualify, it then gives you another long menu.  If you press 0, it places you in a wait queue for a human.  The wait time typically appears to be in excess of 5 minutes.

I noticed that the number of people in your family can’t be more than 8.  If you enter 9 (or any double digit number), you are shuffled back to the main menu.   This looks like a glitch in their logic, since families with more than 8 members do exist and would appear to be eligible for State-provided health insurance.

After entering my family information and my income, I was connected to a human (after a wait of about 5 minutes).  During the wait, the system asked me for additional information regarding what I was calling about.  When I finally did get to an agent, she was polite and seemed knowledgable.  None of the information that I had provided to the system was passed to the agent and I had to start all over again. 

 

Dr. Walter Rolandi examination of Commonwealth Choice Customer Service:          

Patient:                                  Commonwealth Choice

Task:                                     Determine eligibility for universal healthcare plan.

Toll Free Number:              1-877-623-6765

 

The Examination Session

Technically, it was not possible to know whether the task was actually complete.  After I answered a few questions, the system transferred the call to a representative, presumably because my answers implied that I was not eligible for the plan.  Only one call was required to navigate to this point and doing so required 124 seconds to complete.  The entire call lasted 133 seconds. 

 

Call Details:

The system is a highly structured DTMF only application.   The user experience begins with a brief greeting followed by about 20 seconds of pointless introduction and a web ad.  Then follows a 14 second Spanish option.  After selecting “English”, an extremely wordy 6-item, 46 second Main Menu is heard.   DTMF choice 5 began with:

 

If you are unsure whether you are eligible….

 

I entered DTMF 5.   The system then asked for the number of adults and children living in the household.  It apparently uses this number to calculate the household income breaking point for the number of people one enters.  The system then asked a yes/no question as to whether the household income exceeded a particular amount.   After answering, “yes”, I heard the transfer message.

 

Thus, the system required over two minutes to obtain three simple bits of information and transfer the call.

 

The Diagnosis

1.      System prompts are far too wordy.

2.      IVR clichés such as a web ad and a Spanish option are present.

3.      Many prompts are clearly unnecessary.

4.      It requires far too much time to perform a presumably simple sorting process.

5.      Overall production quality is unprofessional.

6.      Production quality of the concatenated income amount is all but embarrassing.  It uses mixed male-female voices and obviously inappropriate intonation.

 

The Prescription

1.      Get rid of the wed ad and Spanish option.

2.      Chop the Main Menu to three, succinctly stated items.

3.      Use professional voice talent.

4.      Use the same voice talent for all prompts.

5.      Eliminate or trim down any unnecessary prompts and prompt content.

 

The Prognosis

Probably all right.

 

 

Nov 6 2008

Walt and Walter’s VUI View: NSTAR

The Caller’s View from Walt:

I called NSTAR to obtain information on my account balance.  NSTAR has implemented a speech-enabled attendant.  It worked pretty well.  It asked me what I was calling for and it recognized my saying account information.  It then took my account number.  This worked fine is I said each digit individually but did not recognize it when a said it as a natural number.  When it didn’t recognize the account number, it defaulted to Touch-Tone right away.  This is a much better approach than the “did you say ….” or asking you to say it over again.   I obtained my account balance in well under a minute.  I had a question about it and requested to talk to a human.  I was connected to a human in less than a minute.  She did not have my account number available.  NSTAR has implemented a pretty good self-service capability.  It provides a quick-&-easy experience to the caller.  They then spoil it all by not passing the information to the agent.  Whatever time is saved by having a quick-&-easy implementation is now a lost by having to repeat everything to the agent. 

I attempted to talk to a human after normal business hours.  I was informed that they were closed and that I should call back when they are open.  No voice mail or offer to call back the next day.

Dr. Walter Rolandi examination of NSTAR Customer Service:    

 

Patient:                                  NSTAR Power Company

Task:                                     Obtain current account balance  

Toll Free Number:              1-800-592-2000

 

The Examination Session

Just one call was required to complete the task.  The task was completed in a mere 42 seconds and the entire call lasted only 83 seconds. 

 

Call Details:

The system is a speech recognition application with an “open ended” (“How may I help you?”) Main Menu.  The user experience begins with the caller hearing:

 

Hello.  Thank you for calling NSTAR. 

For quality purposes, your call may be recorded.

I am an automated assistant.

How may I help you?

 

I responded, “I need to get my current account balance”.   After about a three second pause, the system replied:

 

Account balance and recent payments.

 

It was unclear whether the system response was intended as a confirmation or a question.   The intonation of the prompt implied that it was a question but this was far from obvious.  Confused, I said nothing.  After another 2.5 second pause, the system continued:

 

Please hold while I look up your account.

OK. 

What account number are you calling about?

 

I then stated the 11-digit account number I was to use to which the system responded:

 

OK.

Your account balance is $289.79.

 

At this point, my task was complete.  The system proceeded however with an additional 14-plus seconds of unasked for information about the account.  The additional account information was followed by a legalese disclaimer and a series of yes/no questions offering to provide even more additional account-related information or services, all of which I declined.  The system then thanked me for calling, read me a web-ad, said, “Goodbye” and hung up.

 

The Diagnosis

1.      This system appears to be very healthy indeed.

2.      Speech recognition was perfect.

3.      Task completion was quick and easy.

4.      Some system feedback was less than clear. (“OK.”)

5.      The production quality of a number of prompts is questionable.

6.      Some prompts sound garbled or electronically distorted. 

7.      The functionality of some prompts is questionable (“I am an automated assistant.”)

8.      Some prompt content could be trimmed away.

9.      Small silences could be reduced.

 

The Prescription

1.      Clean up distorted prompts and rerecord as necessary.

2.      Provide more meaningful user feedback.

3.      Eliminate the few unnecessary prompts and prompt content.

4.      Try to eliminate the pauses that follow user speech.

 

The Prognosis

Excellent.

 


 

Walt and Walter’s VUI View: Central Maine Power (CMP)

The Caller’s View from Walt:

I called CMP to obtain information on my account.  They have a structured TouchTone self-service.  Lots of long-winded instructions.  They tell you about their web site and to pay attention since their menu has changed.  I requested my account balance via the automated service.  It took over 2 minutes to accomplish this.  This was really tedious to use.  After getting my account balance, I requested to be connected to a human.  This was accomplished quickly (less than a minute).  The agent had my account information available and was quickly able to answer the question that I had.  The availability of CTI compensated for the laborious self-service.  The only benefit of self-service is to access information after hours when humans are not available.  During business hours, agents appear to be readily available and provide a much more pleasant and quicker response than the self-service.  No point in using the self-service during normal business hours. 

I attempted to talk to a human after normal business hours.  I was informed that they were closed and that I should call back when they are open.  No voice mail or offer to call back the next day.

 

CMP

Dr. Walter Rolandi examination of Central Maine Power Customer Service:   

Patient:                                  Central Maine Power Company

Task:                                     Obtain current account balance  

Toll Free Number:              1-800-750-4000

 

The Examination Session

One call was required to complete the task although another call was made to determine whether the system supported dial through (it does).  The task required 126 seconds to complete.  The entire call lasted 145 seconds. 

 

Call Details:

The system is a highly structured DTMF only application.   The user experience begins with 46 seconds of preemptive instructions, web ads and additional unrequested information.   It is too wordy to transcribe.  Eventually, the system instructs the caller:

 

If you’re calling from a touch-tone phone, please press one.

 

Upon compliance, the system continues:

 

With this automated phone system, you may use several self-service options.

For self-service options, press one.

 

Upon compliance again, the system read out the company’s variant of the IVR cliché:

 

Please listen carefully before selecting from one of the four options.

 

The first option turned out to be “check your account balance”.  After listening to the entire menu, I entered DTMF1.   I was then prompted for my account number which I keyed in.  The system then laboriously read the account number back, requiring me to confirm it with (yet another) DTMF1 response.  The confirmation process began with the pointless assertion:

 

We have recorded your 13-digit account number as….

 

At this point, I was finally read the account balance followed by some additional unrequested account information.  I was then told that additional account services were available by pressing (guess what) DTMF1.  My task complete, I hung up at this point.

 

The Diagnosis

1.   This system is sadly afflicted with many of the telltale symptoms of poor DTMF IVR design.

2.   It is tedious and laborious to use.

3.   It is full of unnecessary prompts and prompt content.

4.   It appears at times to be illogically organized.

5.   It requires what appear to be unnecessary or pointless steps.

6.   Task completion is painstakingly slow.

7.      Overall production quality is sub par.

8.      The confirmation strategy is irrational and unnecessary.

 

The Prescription

1.      Get rid of the entire 46-second preemptive preamble.

2.      Streamline the organization of the system in order to make its functionality more immediately obvious to the caller.

3.      Eliminate any dialog state that is not absolutely required.

4.      Eliminate or trim down any unnecessary prompts and prompt content.

5.      Eliminate the confirmation strategy.

6.      Adopt industry best practices for DTMF design.

 

The Prognosis

Reasonably good.

 

 

 

 

Sep 7 2008

Walt and Walter’s VUI View: USCIS

The Caller’s View from Walt:

I needed to locate the nearest office of the US Citizenship & Immigtation Service (USCIS).

USCIS is a government agency comprised of 15,000 federal employees and contractors working in approximately 250 Headquarters and field offices around the world.

 

They are responsible for the administration of immigration and naturalization adjudication functions and establishing immigration services policies and priorities.

 

I called the number that was listed on the GetHuman web site (800-375-5283).  After the press 1 for English option and being told to “please pay attention since the menu had changed, I ran into the longest main menu that I’d ever encountered.  The prompts just went on-&-on.  It used all of the keys for options and when it was done with all of the options, went on a long harangue about going to the Internet to get all of the information on the options and much more.   It even took the time to suggest that if you did not have access to the Internet in your home or business, then you can go to the local public library to obtain Internet access.  At the conclusion of the IVR session, I was asked if I wanted to take a survey.  I took it and it proceeded to ask about 10 questions on what I thought of the IVR system.  The US CIS appears to be trying very hard to provide a good deal of information to their callers and failing miserably.  They somehow think that using 10 words when it can be done quite well with a single word or two, is an effective way of communicating.  This is one of the most unnecessarily verbose implementations that I have ever run into.  It is so tedious that it is virtually impossoble to use without becoming highly irritated.

 

Dr. Walter Rolandi examination of US CIS Customer Service:     

 

Patient:                                  US Citizenship & Immigration Service

Task:                                     Obtain location of nearest US CIS office  

Toll Free Number:              1-800-375-5283

 

The Examination Session

Only one call was necessary to complete the task.  The task required 213 seconds in total or approximately three and a half minutes. 

 

Call Details:

The system is a highly structured DTMF only IVR.  Calls are answered with a 20 second greeting that includes an unnecessarily wordy and poorly thought out Spanish option.   The male voice talent is polished, professional and of fine production quality.  Selecting the English language route leads to the familiar IVR cliché, “Please listen carefully.  The options on our Main Menu have changed”.   This is then followed by what appears to be the longest, most verbose Main Menu I have ever encountered. 

 

Normally, the Call Details section includes transcription of the patient IVR dialog through the termination of the Main Menu.  In this special case, transcription would require more time than I have to devote to the examination.

 

Suffice it to say that the Main Menu includes no less than ten (10) choices, all expressed in pointlessly verbose language and many of which include preemptive prompt content.   Additionally, the Main Menu includes a very lengthy web ad.  The Main Menu ultimately offers an option to press “*99 to take a 30 second survey”.

 

Navigating to the end of the system’s Main Menu required 121 seconds (over two minutes).  After this arduous journey, I entered DTMF4 to find the nearest US CIS.   This lead to a 27 second submenu wherein the user is required to enter DTMF1 to obtain the location information that was previously made available as choice four on the Main Menu.  Entering DTMF1 lead to a request for the caller’s zip code.  Upon entering the zip code, a nine second silence followed by a readout of the requested information.  The office location information sounded like a locally recorded insert.  A female voice talent is used.

 

After the location information was readout, the “system voice” returns with some more verbose instructions about how to repeat the information, return to the Main Menu and hang up.

 

The Diagnosis

1.      The system could serve as a textbook example on what not to do when designing an IVR.

2.      Overall production quality is good.

3.      Spanish option is too long and poorly planned out.

4.      Main Menu is far too long.

5.      Prompts are far too verbose.

6.      Preemptive prompt content abounds.

7.      Unexplained silence follows zip code entry.

8.      Pace is slow.

9.      System is unbelievably tedious and time consuming to use.

 

The Prescription

1.      The VUI Surgeon’s Scalpel is desperately required

2.      Extract the Spanish option or rethink its design and length.

3.      Cut the number of menu choices.

4.      Debride all prompts of unnecessary prompt content.

5.      Excise all preemptive content.

6.      Investigate the silence between zip code entry and response.

 

The Prognosis

Grim without radical surgery.

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

Aug 7 2008

Walt and Walter’s VUI View: AARP

The Caller’s View from Walt:

I needed to obtain a replacement AARP membership card.  I called the main AARP number (888-687-2277).  After asking me if I was a member (I responded that I was) it presented a four item menu to me.  One of the menu items was Membership Card, so I said “Membership Cards”.  It then asked me for my 10-digit membership number.  I responded by saying “I don’t know it”.  It then came back and said: “Did you mean “I don’t know it”. Please say “yes” or “no””.  I said “yes” and it responded with: “please hold for the next available representative”.

The inappropriate error recovery after I said “I don’t know it”, ruined a really good call experience.  No reason exists to obtain a confirmation that I really said that I didn’t know my membership number.  Any response other than me providing the number should have caused the system to simply go ahead and transfer me to a CSR.  Jerking the caller around with a “did you mean?” is just wrong.

When I reached a CSR, she did not know what selection I had made at the menu and I had to repeat that I needed to obtain a replacement membership card.

I made 4-5 calls and each time it asked me whether I was a member or not.  The system does capture the CallerID but appears to do very little with it. 

 

Dr. Walter Rolandi examination of AARP Customer Service:       

Patient:                                  AARP Service IVR

Task:                                     Find out membership cost

Toll Free Number:              888-687-2277

 

The Examination Session

Two calls had to be made to the system before concluding that it apparently does not support the task.  There is no obvious way to determine membership costs using the system.  The two calls consumed approximately 100 seconds. 

 

Call Details:

Calls are answered with a 2 second audio-icon followed by cheerful and professional sounding female voice saying: 

 

Welcome to AARP!

This call may be recorded for quality.

After approximately a 2.5 second pause, the system states:

 

Please say yes or no.

Are you or the person you’re calling for a member of AARP?

 

Not being a member, I answered, “No” after which the system told me:

 

Please hold for the next available representative.

 

Not wishing to bother “the next available representative”, I hung up.

 

I immediately called back.  On this call, I answered, “Yes” to the initial question and heard the following: 

 

Main Menu

Please say if you’re calling for AARP products and services

Insurance

Membership cards

Or to check expiration date

 

I ventured a guess as to where I might find the cost of membership and said, “Products and services”.  After a 3.0 second silence, the “products and services” menu was read.  There was nothing on the menu that suggested membership costs although its fourth (and last) option was to say, “Something else”.  Saying, “something else” lead to another menu listing several other services, mostly having to do with travel planning.

I returned to the Main Menu and wondered about a bit more but eventually concluded that membership cost information was probably not available through the system.

 

The Diagnosis

1.      The system seems to enjoy good health.

2.      It is well designed and easy to navigate.

3.      Overall good production quality is good.

4.      Its menus are short and well land marked.

5.      Pace is slow but this is probably by design, given that the system primarily serves the aged.

6.      Some silences are probably too long

7.      Access to a human is never far away.

 

The Prescription

1.      Investigate the silences between prompts and trim them to a universal length

2.      Play audio-icon during (not before) the welcome prompt

 

The Prognosis

Excellent.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

July 9 2008

Walt and Walter’s VUI View: The Internal Revenue Service (IRS)

The Caller’s View from Walt:

I needed a Form 8109-C for my quarterly tax deposits. I called the main IRS number (800-829-1040).  After a long introduction, I finally got a menu of four choices: 1st was to obtain information on the rebate.  None of the other choices mentioned obtaining tax forms, so I selected an option that offered to provide information on filing my taxes.  After that, the 1st menu choice was:To order tax forms or publications, press 1”  Thinking that I had found it, I immediately pressed 1. It responded with: “If you know the the form or publication that you need, press 1, if you don’t the form or publication, press 2.”  I responded with a 1.   The system then responded with a long monologue about the status of the rebate payment.  To confuse things, this time they referred to it as the “stimulus payment”.  At this point I bailed out of the IRS IVR and (after a 10 minute wait) was connected to a human that took my order for the form that needed.

 

Not being able to order a form via the IRS IVR is awful.  It is a waste of my time as well as the time of the agent.  It appears as though they recently added the rebate information into the IVR and this somehow managed to mess up the form request service.  Beyond this, the IRS self-service is just very poorly done.

 

Dr. Walter Rolandi examination of Internal Revenue Service (IRS):        

Patient:                                  Internal Revenue Service IVR

Task:                                     Order a 1040 form

Toll Free Number:              800-829-1040

 

The Examination Session

Three calls were made to the system before I abandoned the task.  There is no obvious way to order forms using the system as I found it on the examination date.  The three calls consumed approximately six (6) minutes to complete. 

 

Call Details:

 

A Main Menu (of sorts) appears after approximately 26 seconds of the above preliminaries.  It begins:

 

If you are calling for information about the tax rebate, press 7.

For recorded tax topics, press 1.

For questions about your refund, press 2.

For all other automated services press three.

To hear these options again, press 9.

 

Thinking “all other automated services” was the logical choice, I entered DTMF 3.  This lead to several DTMF beeps (apparently a line transfer), five seconds of silence and an instruction to enter either a social security or employer number in order “to access your account information”.  Confused, I did nothing.  Approximately six seconds later, the system repeated the request.  Failing to see what my social security number had to do with ordering a 1040 form, I hung up.

 

I immediately called back, again experiencing everything above but paying special attention to the Main Menu, assuming that I had reacted to it incorrectly on the initial call.  Again I heard:

 

If you are calling for information about the tax rebate, press 7.

For recorded tax topics, press 1.

For questions about your refund, press 2.

For all other automated services press three.

To hear these options again, press 9.

 

Still confused but wanting to be sure, I entered DTMF 9 to hear the options again.  Astonishingly, the options were not repeated.  Female #1 simply restated the greeting, normal business offices and the web ad followed by Female #2 doing the same but in Spanish.  The system then promptly hung up.

 

Again, I immediately called back and navigated through to the Main Menu.  This time I entered DTMF 1 to see if “recorded tax topics” could lead to information about ordering forms.   This lead to another (apparent) transfer, a number of preemptive prompts and, eventually, an announcement that “there are seventeen (17) main categories for which there are topics”.   After listening to the first seven topics, I hung up. 

Note that it is possible that the system provides information about form ordering somewhere between topic eight and topic seventeen.

 

The Diagnosis

1.      System is deathly ill.

2.      It exhibits repeated violations of the de facto DTMF standard.

3.      Overall organization is illogical.

4.      Jargon is present.

5.      Spanish option is present.

6.      Web ads are present.

7.      Main Menu is illogically laid out.

8.      One submenu had 17 (!) items.

9.      DTMF entry is awkwardly (if not stupidly) used of on 17-item submenu.

10.  Pointless preemptive prompts are present.

11.  Pace occasionally seemed slow.

12.  Overall good production quality is wasted on the senselessness of the system’s organization and function.

 

The Prescription

1.      Completely reorganize the entire system.

2.      Adopt de facto DTMF standard.

3.      Eliminate jargon.

4.      Eliminate Spanish option.

5.      Eliminate web ads.

6.      Eliminate preemptive prompting.

7.      Speed up overall interactive pace.

 

The Prognosis

Without the most drastic intervention, very poor indeed.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

 

 

June 9 2008

Walt and Walter’s VUI View: The Annual Credit Report Service

 

The Caller’s View from Walt:

An amendment to the federal Fair Credit Reporting Act requires each of the nationwide consumer reporting companies – Equifax, Experian, and TransUnion – to provide consumers with a free copy of their credit report, at their request, once every 12 months.  I called the Annual Credit Report Service to order a credit report.  This service is provided by the three (3) major consumer credit reporting companies.  I had previously ordered a credit report from this service, so I was a repeat caller.  You can order three (one from each of the credit reporting companies) reports each year, so most calls are repeat calls, since it makes sense to not obtain the three reports at the same time.  The service is oblivious to this and always assumes that you are calling for the first time.  It forces you to listen to an excessively long and unnecessary monolog.  Overstrike is ignored.  The voice relentless continues to babble on describing the service and what it does.  It describes in detail all of the ways in which you can order a credit report such as the Internet and mail.  The system somehow thinks that the consumer that used the telephone to request a fee credit report, needs to be told over-&-over again what the service is, how it works and the alternate ways that they can request a credit report. Despite using your phone number to identify who you are and what your address is, historical information is not used.  With your phone number, it would be easy to determine that you are a repeat user and that the excessively long monolog can be by-passed.   They do not provide any human backup so the only option is to just suffer through the horribly implemented self-service.  The intent of the awful implementation appears to be to discourage consumers from requesting a credit report.

 

Dr. Walter Rolandi examination of Annual Credit Report Service:                                   Annual Credit Report Service

Task:                                     Order a report

Toll Free Number:              877-322-8228

 

The Examination Session

Only one call was necessary to complete the task.  The process took just over six (6) minutes however.  It was at times, laborious, painstaking and annoying.

 

Call Details:

Normally, specifics of the interactive dialog are included in this section.  However, given the length of the call, no effort to transcribe its entirety was made.  Some specific pieces of the dialog are reviewed.

The system answers and the caller hears an eight-plus second initial greeting spoken by a professional sounding female voice talent.  It then launches into a 24 second monologue describing what the system is and what it exists to do. 

 

Eventually, the system instructs callers calling from their home phones to “… press or say, one.  If not, press or say, two”.  This is the first incidence of an unnecessarily wordy method for negotiating either-or logic branches within the application.   Variants of the template unfortunately are unremittingly repeated throughout the interaction.

 

Upon responding, the caller is treated to a 15 second “web ad” followed by a five second prompt offering to provide instructions on how one might submit their credit report request in writing.   At a point soon thereafter, one finally arrives at the dialog state where the process of actually making a request begins.  In my case, navigating to this state had required 99 seconds.

 

After an additional four minutes and 25 seconds, I successfully completed the credit report order.  The process included some notable speech recognition failures.  Among them were one failure to recognize the utterance, “no” in response to a “yes/no” question and two failed attempts to accurately recognize the spelling of my name.  

 

Overall, the interaction was riddled with lengthy if not actually unnecessary prompts.   Many were preemptively instructive and many of the preemptive prompts were needlessly repeated throughout the session. 

 

Some user inputs seem to be confirmed unnecessarily.  The system has a pointlessly verbose and questionably effective error recovery strategy and, at times, the system prompts appear to suggest that it has conversational abilities that it clearly does not possess.

 

The Diagnosis

1.      This system suffers from neuro-linguistic infirmities.

2.      It seems to take every opportunity to say something that seldom if ever needs to be said.

3.      It also seems to say things that could be easily said with just a few words in far too many words.

4.      There are many preemptive prompts.

5.      Its time-to-task-completion is painfully slow.

6.      Overall production quality is good.

7.      Speech recognition should probably be tuned.

8.      Some prompts are artificially conversational.

9.      System response times are occasionally slow.

 

The system’s health could be dramatically improved with some effective neuro-linguistic programming.

 

The Prescription

1.      Excise all unnecessary prompts and prompt content.

2.      Eliminate preemptive prompting.

3.      Streamline the time-to-task-completion.

4.      Investigate speech recognition accuracy issues and tune accordingly.

5.      Tone down the suggestion of “conversational” abilities.

6.      Investigate response time issue.

 

The Prognosis

Very good.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

 

May 9 2008

Walt and Walter’s VUI View: The American Airlines Customer Support Line

The Caller’s View from Walt:

I called American Airlines to locate a flight from Boston to New York.  I requested “Flight and Gate information” from a four item menu.  Even so, it came back and confirmed that I had requested Flight and Gate Information.  I finally nailed down the flight that I wanted with it repeating everything that I said and asking for confirmation of each item.  I then attempted to get to a CSR to book the flight.  Waited in a queue for about five minutes.  It did not tell me the wait time or offer to call me back.  When I finally did get to a CSR, she did not have the information that I had provided to the flight information system and I had to provide it to her all over again.

American Airlines makes using the self-service an unpleasant experience. It has long and often unnecessary prompts and parrots virtually everything back to you.  When you get to the CSR, you need to repeat everything to the CSR again since nothing is passed from the self-service to the CSR.

 

Dr. Walter Rolandi examination of American Airlines customer support line

Patient:                                  American Airlines

Task:                                     Find out if service available from one city to another.

Toll Free Number:              800-433-7300

 

The Examination Session

The call was answered by a directed dialog speech recognition system.  The call was apparently transferred to another speech recognition system and eventually back to the original system that appeared to answer the call upon saying, “Main Menu”.  Only one call was required to complete the task although it did entail two “loops” through the system.  

 

Call Details:

The system answers the call with a pleasant female voice talent saying:

Thanks for calling American Airlines.  Para Español, diga, “Español”.

These prompts are immediately followed by:

Main Menu.

Please say reservations, flight and gate information, Advantage services or more options.

Upon my saying, “Reservations”, the system cheerfully responded:

Ok. Reservations.  If that’s not right, say, “Go back.”

After a pause of almost two seconds, the system continues:

Now which one would you like? 

Fares, new reservation, existing reservation or flight and gate information.

After clearing responding, “Flight and gate information”, the system replied:

I think you want, “Flight and gate information”.  Is that right?

Answering, “Yes”, the system continued with:

The next time, you can get here more quickly by saying, “Flight and gate information” at the Main Menu.

This call may be recorded.

Next came three seconds of silence followed by four seconds of music-on-hold and an apparent call transfer to another system:

Welcome to the flight information system for American Airlines, American Eagle, American Connection and other American marketed code share flights.

The intonation and pace of this prompt were unnatural, suggesting that prompt content had been electronically edited to include or delete some information.  The system continues:

To get you up to date on departure and arrival information, I’m going to ask you a few questions.   First, what’s the flight number?

A pause of .82 seconds follows whereupon the system announces:

It’s Ok to say, “I don’t know”.

I said, “I don’t know”, the system went silent for approximately two seconds and then carried on:

Ok let’s find out what flight you want.  What’s the departure city?

“Boston”, said I.

Next!  Tell me the arrival city?

“Chicago”.

Thanks!  Now is that for arrival or departure info?

“Departure”.

And finally, around what time does that flight depart?

“Noon”.

Lemme confirm that….

A laborious restatement of all this information subsequently follows after which the system located a relevant flight.   The entire process required two minutes and twenty-three seconds to complete.

 

The Diagnosis

1.      This system is reasonably healthy but it is not without some infirmities.

2.      It suffers from slight production quality issues.

3.      Volume, intonation and pace vary significantly from prompt to prompt.

4.      At times, pace is so fast that it makes comprehension difficult. 

5.      Interaction is artificially conversational.

6.      At least one pause invites false starts (i.e., “First, what’s the flight number?” (.82 second pause)  “It’s Ok to say, “I don’t know””.

7.      Persona borders on the annoying.

8.      Many prompts are unnecessarily long.

9.      Several prompts contain useless content (i.e., “To get you up to date on departure and arrival information…”)

10.  There are some preemptive prompts and “hints” present.

11.  Prompts contain jargon (i.e., “American marketed code share flights”)

12.  Some IVR Clichés are present.

13.  System response times seem inconsistent.

The system’s health could be greatly improved with some minor surgery.

 

The Prescription

1.      Clean up production quality issues with volume, intonation and pace.

2.      Tone down the animation in the persona.

3.      Re-examine pause and timeout practices.

4.      Speed up interaction.

5.      Eliminate IVR Clichés, jargon, preemptive prompts and “hints”.

6.      Trim all prompts of unnecessary content.

7.      Investigate response time issue.

 

The Prognosis

Quite good.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

Apr 12 2008

Walt and Walter’s VUI View: The Tufts Healthcare Customer Support Line

The Caller’s View from Walt:

I needed to obtain a 1099HC from my health insurance company to prove to the state of Massachusetts that I had health insurance.  I called the Tufts Healthcare customer support line.  I was greeted by a voice that sounded identical to the Amtrak Julie persona.  So much for unique personas.  I was calling at 5:30 in the evening.  This turned out to be outside of normal business hours.  It dumped a menu at me that gave me the choice of 1) either checking my bill; 2) obtaining information re my pharmaceutical coverage; or 3) or filing an appeal of a denial of services.  After the menu, which was way to verbose, it then announced that they were closed and that if I wanted anything else, I should call back when they were open for business.  Imagine that. Not even voice mail.

I called back the next day.  Got a 6 item menu for identifying the type of caller that I was.  This could have been eliminated by looking at my CallerID.  Next was a menu that had claimed to have 7 choices and then gave me 8 choices.  What I was calling about was not one of the choices, so I chose the “none-of-the-above”  choice.  When I finally reached a CSR, none of this information was passed which meant that I had to provide it all over again.   By not having CTI implemented, Tufts Healthcare is wasting my time as well the time of their CSR.  

 

Dr. Walter Rolandi examination of Tufts Healthcare customer support line

Patient:                                  Tufts Healthcare

Task:                                     Obtain name, address and phone number of an eye care provider

Toll Free Number:              800-701-9000

The Examination Session

Initial system branches off to at least one other system.   First system is a “press or say” directed dialog IVR.  Second system supports slightly broader speech recognition but also a directed dialog.   System had to be called four times in order to discover how to complete the task.  

Call Details:

The system answers the call with a female voice talent saying:

Thank you for calling Tufts Health Plan Medicare Preferred.  For quality purposes, calls may be monitored and recorded.  Please listen carefully to the following options as the menu has changed.

After this greeting and IVR cliché ordeal is complete, (almost 11 seconds), one hears this monstrous 30-plus second Main Menu:

            If you are a provider or are calling on behalf of a provider, press or say one.

 

If you are a member of our HMO plan, press or say two.

 

If you are a member of our PPO plan, press or say three.

 

If you are a member of our Private Fees for Service Plan, press or say four.

 

If you are unsure of your plan type, please look in the upper right hand corner of your ID card.

 

If you’re interested in becoming a member of Tufts Health Plan Medicare Preferred, press or say five.

The system then pauses a mere 1.5 seconds before announcing:

Please make your selection now.

I had been prepared to say, “Two” since I was calling as an HMO member but the “Please make you selection now.” prompt confused me and I paused.  After approximately 3.0 seconds, the system continued:

If you’re calling from a rotary phone, please stay on the line. To repeat this menu, say repeat or press star.  To end this call, simply hang up.

Almost an entire minute (59 seconds) has now passed since the system answered the call.   It finally permitted me to say, “Two”.  After another 2.0 seconds and without any additional feedback, the system launched into a 63 (!) second submenu which began:

Please listen carefully and select one of our seven options….

This prompt was also punctuated with the utterly useless statement:

Please make your selection now.

It was not immediately clear which menu choice I should indicate in order to locate an eye care provider.   One erroneous choice was guessed after which I hung up.  I then called back two more times, indicating option “three” because it said something about “Vision Care” (?).  Both times the system responded that my call was being transferred to “Eye Med Member Services” (?) and, assuming that a human would soon answer the call, I hung up.  On the fourth and last call, I did not hang up.  After approximately 5 seconds, a line was heard ringing.  The ringing lasted for over 9 seconds whereupon another system appeared to answer:

Welcome to Eye Med Vision Care (?).

There are several things that I can help you with.  Please say one of the following:

Provider locator, plan benefits, claim forms or claim status.

Upon saying, “Provider locator”, the system then read an 11 second prompt describing my vision plan and requesting a zip code where to look for providers.  I spoke the zip code, “01720” after which the system said,

Hold on while I look that up.

The system soon came back saying that it hadn’t been able to locate any provider within 100 miles of the zip code I that I had provided.  What followed was a series of nonsensical instructions that required me to return to the Main Menu in order to say another zip code.  I eventually muddled through, finally completing the task in just over 5 minutes and 15 seconds!

The Diagnosis

1.      This system is deathly ill.

2.      The system (or systems) supports inconsistent interactive modalities.

3.      System attempts to automate far too many tasks.

4.      Prompts are unbelievably long.

5.      Preemptive prompts are abundant.

6.      Menus have too many options and are far too wordy.

7.      The system is painstakingly slow to use.

8.      Feedback is often insufficient.

9.      Logical organization of the system is occasionally unapparent.

10.  Timeout timers are often inappropriate.

11.  Speech recognition accuracy seems sub par.

12.  A bevy of IVR Clichés is present.

13.  System response to input seems slow.

14.  Production quality is low-to-medium.

Performing this seemingly simple task using this system (or systems) revealed the severity of its ailments.   Only the most profound interventions can save this patient.         

The Prescription

1.      Bust up the various components into smaller, more manageable and functionally rational systems.

2.      Adopt a radically different design philosophy.

3.      Employ a consistent interactive modality.

4.      Provide meaningful feedback.

5.      Speed up interaction.

6.      Improve ASR accuracy.

7.      Eliminate IVR Clichés.

8.      Shorten menus.

9.      Trim all prompts.

10.  Improve production quality.

 

The Prognosis

Without fundamental and radical surgery, I am afraid there is little hope.

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

Mar 7 2008

Walt and Walter’s VUI View: The Charter Communications Customer Support Line

 

The Caller’s View from Walt:

 

I attempted to find out what the cost and availability of the Charter Communications Triple-play package was by calling the telephone customer support number (866-472-2200). After a brief greeting, and the press 1 for English prompt, it asks you to enter your phone number, which I promptly did.  Charter Comm’s appears to ask you to provide your phone number for no purpose other than to irritate and waste the time of the caller.  They do nothing with the number.  The menu offers me four options.  They assume that I’m already a subscriber since they are about billing, scheduling a technician appointment or obtaining technical assistance.   They had forced me to provide my phone number from which they should have easily been able to determine whether or not I was a subscriber.    I selected the 4th menu item which was for purchasing services from Charter Communications.  The options were a new subscription and a pay-per-view movie.  Again, offering me the pay-per-view option was inappropriate for them to do for a non-subscriber.   I was next connected to a CSR.  I asked him if he had my phone number or knew what I had requested.  He said that he did not.   He also indicated that he handled all types of calls from billing to new services.

 

These people want to be your telephone company.  Yet, they don’t support rather rudimentary telephony functions such as CallerID or CTI.   Why they bother to waste the caller’s time listening to a menu and responding to it, totally escapes me.   You are routed to the same CSR no matter what you input, so the entire IVR process serves no purpose other than to waste the caller’s time.

 

Dr. Walter Rolandi examination of Charter Communications customer support line

Patient:                                   Charter Communications

Task:                                      Availability and price of broadband service

Toll Free Number:    866-472-2200

 

The Examination Session

System is a DTMF-only direct dialog IVR.  It was called three times.   Apparently it does not support automated broadband availability and pricing information.   Following the “new service” path eventually leads to a transfer. 

 

Call Details:

The system answers the call with a female voice talent saying:

 

Welcome to Charter Communications.

This Classic IVR Cliché immediately follows the greeting:

 

For quality assurance purposes, your call may be monitored or recorded.

 

Then comes a Spanish option:

 

To continue in English, press one.  Para continuar en Español, oprima el numero dos.

 

Once one’s language preference has been discerned, the system then asks the caller to enter his “10-digit home phone number beginning with area code now”.

After complying and the passage of approximately three seconds of silence, the Main Menu emerged.  Just over 22 seconds had elapsed since the call was answered.

 

For billing questions or account balance information, press one.

If you’re calling in reference to a scheduled appointment, press two.

If you require technical assistance, press three.

To order new service, to add additional services or to transfer your service, press four.

To disconnect or remove services, press five.

 

The Main Menu required over 20 seconds to be read extending call length to over 40 seconds.  Note that all of this time was required before being given the opportunity to indicate why I had called. 

 

Main Menu choice four seemed the most related to my task.  When I entered DTMF 4, another 14-second menu followed, the first choice of which was:

 

To order new Charter Services, press one.

 

I entered DTMF 1 and after approximately 5 seconds of silence, two loud DTMF beeps were heard.  Ten seconds later, an agent named Diane came on the line.

 

 

The Diagnosis

1.      Spanish option questionably implemented

2.      Prompts are unnecessarily long.

3.      Menus are too wordy.

4.      The Main Menu could be shortened.

5.      The system does not appear to support the task I intended to perform.

6.      IVR Clichés are present.

7.      System response to input seems slow.

 

Far from dreadfully ill, the system could be more accurately described as being, “out of shape”.  It could be “whipped into shape” very easily however and perhaps be further improved with little cosmetic surgery.

 

The Prescription

1.     

2..      Eliminate or redesign Spanish option.

3.      Shorten Main Menu.

4.      Trim all prompts.

5.      Improve production quality (i.e., provide a transfer message instead of two loud beeps).

6.      Speed up system response time.

 

The Prognosis

Very promising.

 

 

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com

 

 

Feb 8 2008

Walt and Walter’s VUI View: The Washington Post Customer Support Line

The Caller’s View from Walt:

We called the Washington Post this month to see how it differed from The New York Times and The Wall Street Journal which we tested previously and found to be wanting. 

I attempted to find out what the cost was to subscribe to the Washington Post by calling the telephone customer support number (800-477-4679). After a brief greeting, it tells you to pay attention since their menu has changed.  You then encountered a menu where the first choice was to start a new subscription which I selected.   The CSR informed me that a one-year subscription to the Washington Post would cost me $187.20.  The on-line version would cost an additional $9.95 per month or $119.40 per year.   

The main menu contained a number of choices.  In addition to the new subscriber choice, it also had other selections like cancel subscription or report a damaged/missing paper.   I tried these out and discovered that they were just wasting my time since you went to the same CSR pool no matter what you chose and the CSR did not know which selection you made so you had to again inform the CSR what you are calling about.

I decided to call back after business hours.  Instead of the option to go to the CSR to start a new subscription, I was instructed as follows:

 “If you’re calling for any other reason, please call back during normal business hours.  Our customer service representatives are available from 6 AM to 6 PM weekdays  7 AM to 12PM on Saturdays or holidays and from 7AM to 2PM  Sundays.”

Wow!  Someone wanting to become a subscriber and The Washington Post doesn’t think that they should capture their phone number or let them leave a voice mail message?  The tiniest business typically does better than this.  They have voice mail at night and will call you back the next business day.

I also called the Washington Post Advertising Hot Line.  (800-765-7678). It greeted me with:

The Washington Post Advertisers hot line is presently closed.  Normal business hours are 9 AM to 5:30 PM eastern time.  Please contact us during those hours or if you have Internet access, please visit our web site at www.ad site, that’s a d s I t e dot wash post  w a s h post dot com. Thank you for calling.” 

These folks are seeing advertising revenue that is dropping like a rock.   Yet they don’t even have a voice mail service where a potential advertiser can leave a message and can be called back the next day.  These people deserve to go out of business.  They are working hard to earn it!

Dr. Walter Rolandi examination of The Washington Post customer support line

The Examination Session

Continuing our evaluation of major newspapers, this month we examined the Washington Post.   For the sake of consistency, we attempted to perform the same task that we had tried when calling both the Wall Street Journal and New York Times: we attempted to obtain the rate for a one-year subscription to the newspaper.

 

I called the system at 800-477-4679.   As was the case with the both the Wall Street Journal and New York Times systems, the information I sought did not seem to be available through the IVR.  Only one call was necessary but several more were made however in order to clarify a question about the main menu grammar.

Call Details:

The system answers the call with a female voice talent and an appropriately succinct greeting:

Welcome to the Washington Post.

The greeting is immediately followed by the Classic IVR Cliché:

Please listen carefully as our menus have changed.

The Main Menu then begins:

To start a subscription, please say, subscription or press one.

To report a damaged, incomplete or missing paper for today, Friday, say, complaint or press two.

To temporarily stop your newspaper with a restart date, say stop or press three.

If you’re calling to cancel your subscription, say, cancel or press four.

For billing enquiries, say, billing or press five.

To speak with a customer service representative, say, service or press zero.

Assuming that the “start a subscription” path would lead to information about subscription rates, I said, “One”.   Two and one half seconds later, the system responded with another IVR Cliché:

Your call is being transferred to the next available representative.

I immediately hung up.  In order to determine whether the system actually understood the digit, “one”, I repeated the experiment several times saying different numbers all of which appeared to be in the grammar.

 

The Diagnosis

1.      Someone was clever enough to include digits in the main menu grammar.

2.      The main menu could be shortened.

3.      The system does not appear to support the task I intended to perform.

4.      The “say or press” prompting style is ineffective and unnecessary.

5.      The “for X, say X” prompting style is unimaginative and annoying.

6.      Most prompts are wordy and unnecessarily long.

7.      IVR Clichés are present.

8.      Response to speech input seems slow.

 

The system is not seriously ill but it would greatly benefit from some cosmetic surgery.

 

The Prescription

1.      Eliminate the IVR Clichés.

2.      Eliminate all “or press” DTMF prompt components.

3.      Reduce main menu choices to three (3).

4.      Trim all prompts.

5.      Speed up speech response time.

 

The Prognosis

Quite good.  Better than most.

 

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

 

 

Jan 7 2008
 

Walt and Walter’s VUI View: The NY Times Customer Support Line

The Caller’s View from Walt:

 

We called the NY Times this month to see how it differed from The Wall Street Journal which we tested previously and found to be wanting. 

 

I attempted to find out what the cost was to subscribe to the NY Times.   After a brief greeting, you are confronted with a horrendous menu.  This is a Touch-Tone system that is pretty poorly done.  Way to many choices to deal with.   I provided all sorts of information about what I was calling about.  I assumed that this information was being used to route me to the appropriate customer service representative (CSR).  I quickly found out that this was just wasting my time since none of the information that I provided to the IVR was passed to the CSR.  I had to repeat everything all over again.  No consistency.  The eight key is used for the repeat function at the main menu but at other menus it is identified as an invalid key.   The NY Times IVR is a pure Touch-Tone service that looks like it was done a decade ago and has been torturing callers for that long. 

 

When I followed the instructions to press 1 for a new subscription, it repeated the instruction all over again twice.   The 2nd time it admonished me to have my credit card ready.  I had to again press 1 for a new subscription.  All of this was a waste, since I had to repeat everything all over again to the CSR This service is pretty brain dead.  Hard to imagine that a major newspaper has their new order processing operation as messed up as this.  This appears to be a death wish.

 

The CSR finally told me what the cost to subscribe was.  They have a relatively complex pricing structure that is dependent on how you are going to pay.  They also offer a lower rate for new subscribers. Although you could argue that this is unfair to existing customers, it wasn’t as offensive as what the WSJ was doing, which I thought was simply dishonest.

 

One positive aspect of the NY Times customer service was that was available 7 days a week and late at night and early in the morning.  The wait to get to a CSR was pretty short.  This was in marked contrast to the service that was provided by The WSJ where it was only available during working hours and instructed customers to call back when they were open.

 

Dr. Walter Rolandi examination of the New York Times customer support line

The Examination Session

 

The patient today was the New York Times.  Given that we had previously attempted (and failed) to obtain the rate for a one-year subscription to the Wall Street Journal, we thought it only fair to ask the same of the New York Times’ IVR.

 

I called the system at 1-800-NYTIMES (1-800-698-4637).  One call should have been sufficient because, (as was the case with the Wall Street Journal system), the information I sought did not seem to be available through the system.  Two calls were made however in order to explore what could only be described as an odd initial user experience.

 

Call Details:

The system answers the call with a female voice talent and an appropriately succinct greeting:

 

Thank you for calling the New York Times.

 

The system immediately followed the greeting with this:

 

For verification, your house or building number is: 8 8 8

and your apartment number is: 4 2 7.

 

If this is correct, press one.  If not, press two.

 

I can only imagine that the system was attempting to identify me (the caller) using ANI.  Since I am not a current subscriber to the New York Times and the information the system asked me to confirm had nothing to do with me, this all seemed very strange and clumsy.   (Note that on the first call, I entered DTMF1 and, on the second call, I entered DTMF2.   The consequences were apparently identical).

A main menu followed:

To order a new subscription, press one.

To report a missed paper, a damaged paper or any other delivery problem, press two.

To make a payment or obtain billing information on your account, press three.

If you are calling to suspend, discontinue or restart your subscription, press four.

If you are calling for any other reason, press five.

To repeat this menu, press eight.

If you are calling from a rotary phone, please stay on the line for the next available customer care representative.

 

Assuming that the “order a new subscription” path would lead to information about subscription rates, I entered DTMF1.  The system responded with:

 

To order a new subscription, press one.

Please have your credit card ready and press one.

For other customer service assistance, press two.

To return to the main menu, press nine.

 

Pressing DTMF1 (again) lead to some stereotypical IVR clichés regarding an imminent transfer to a “customer care representative”.  I hung up during music-on-hold.

 

The Diagnosis

1.      In addition to its physical infirmities, the system suffers from mental disorders.

2.      Initial system behavior appears odd.

3.      Several prompts are clearly irrational.

4.      There are to many choices on the main menu.

5.      The system does not appear to support the task I intended to perform.

6.      Most prompts are wordy and unnecessarily long.

7.      Many prompts contain pointless preemptive content.

8.      At least one DTMF key use practice violates an industry practice.

 

The system does not appear to be very rationally designed.  With all of its “ors and ands”, the main menu contains a cognitively overwhelming eleven items.  Upon electing “to order a new subscription” by pressing DTMF1, the system immediately requires the user to press DTMF1 again.   The DTMF9 key is generally regarded as the de facto standard disconnect or “end call” key, not the “return to main menu” key which is typically “star star”. 

 

The Prescription

1.      Investigate odd initial prompt sequence phenomenon and adjust accordingly.

2.      Rethink all prompts in terms of their rationality.

3.      Reduce main menu choices to three (3).

4.      Trim or remove any unnecessary prompts.

5.      Remove all preemptive prompts or prompt content.

6.      Support de facto DTMF standards.

7.      Speed up the time to task completion.

 

The Prognosis

Reasonable with thorough adherence to all prescribed treatments.

 

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

 

Nov 8 2007

Walt and Walter’s VUI View: The Wall Street Journal (WSJ) Customer Support Line

The Caller’s View from Walt:

My subscription to the WSJ was expiring and needed to be renewed.  I received a notification via the USPS from the WSJ reminding me of this.  The notice indicated that a renewal for one year would cost $219.  They indicated how good a deal this was by showing that it was a $240 saving from the newsstand price.  It was also identified as “special rate” that would expire in two weeks.  I also had a subscription to the on-line version of the WSJ.   Since nothing about the on-line subscription was mentioned on the renewal form, I decided to call.  I called at 7:30AM.  Got a long greeting that went on-&-on and said nothing that a simple greeting that confirmed that I’d reached the WSJ would have done.  It finally stopped babbling away and asked me what I wanted.  It mentioned that it was speech enabled and that I should say what I wanted.  I said “subscription renewal”.  It asked me to repeat this a couple of times.  Finally it said that they could not locate anyone whose name was “Subscription”.  I then got instructions re what things I could ask for.  It listed 18 items that included different magazines, accounting, investor relations, etc. and finally the last one was: “Subscriptions and Customer Service”   Aha!  Now I knew the magic thing to say.  “Subscription Renewal”  or “Subscription” wasn’t good enough.  It had to be: “Subscriptions and Customer Service”.  At that point, I became enthused that my ordeal was nearly over and I said the magic phrase.  It recognized what I had said and spoke it back to.  Next it asked me which publication I was calling about.  I said that it was the WSJ and it again parroted this back to me.  You really feel like telling it to stop repeating everything that I say back to me because it is becoming annoying.  Little point in doing this since it told me which publications that I could request (either The WSJ or Baron’s) and the system should have no difficulty in determining which one was said. After all of this hassle, it then announced that the customer service and subscriptions department was closed and that I should call back during normal business hours.  It does offer an IVR option, which is a Touch-Tone one and the speech capability is no longer available.  No offer to call me back.  No voice mail.  No capturing of my CallerID.   Good grief!  This is that bastion of business that editorializes about proper business practices.  A newspaper subscriber that is calling to renew their subscription and the WSJ tells them to call back when it is more convenient for them.  To make it even worse, it wastes a lot of my time before finally informing me that they were closed. 

 

I called back during normal business hours and went through the entire process again.  It forced me to identify that I was calling about the WSJ (not Baron’s) and that it was it was the print version and not the on-line version.  The option of both the print and on-line (which is what I had) didn’t exist, so I selected the print option.  It again confirmed what I’d said by parroting it back to me and then transferred me to a customer service representative (CSR).  The first thing that the CSR did was ask me just what I was calling about.  All of the time wasted in telling the IVR system what I was calling about was for naught.  Need to start all over again.  After telling the CSR that I wanted to renew my subscription, I was informed that I should really go to the WSJ web site since it would only cost me $79.00.  Wow!  The newspaper industry is struggling with competition for advertising revenue from the Internet.  They seem to be going out of their way to accelerate their demise.  The WSJ renewal process appears to be designed to screw it’s historical subscriber base.  Loyal customers are tricked into paying almost triple what a new subscriber would pay.    

 

Dr. Walter Rolandi examination of the Wall Street Journal customer support line

The Examination Session

 

Our patient this month was the Wall Street Journal.  My examination task was to call the system to obtain the rate for a one-year subscription.  The system can be reached at 1-212-416-2000.  Only one call was made because the information I was asked to obtain did not seem to be available through this system.  Please note that this may be related to the fact that the call was made on a weekend.

 

Call Details:

The system answers the call with a female voice talent who greets the caller to a 13 second introduction to Dow Jones’ “speech enabled automated operator”.  After a 1.5 second pause, the system proceeds:

 

If you know the first and last name of the Dow Jones employee you wish to speak with say, “Employee directory”.

 

Otherwise, please ask for a Dow Jones department, publication or service.  Some of your choices are: The Wall Street Journal; Barrons; the Online Journal; Dow Jones Newswires; subscriptions and customer service or advertising.

 

For a complete listing of departments and services say, “What can I say?”  For immediate live assistance say, “Operator” or press zero.

 

After considering the many choices, I said, “Wall Street Journal”.  The system then replied:

 

The Wall Street Journal!

Which service or department are you looking for?  You can ask for any one of the same categories that appear in the printed version of the Wall Street Journal.  Some of your options are: news department, advertising, subscriptions and customer service.  For an entire list of available departments and services say, “What can I say?”.  For immediate live assistance say, “Operator” or press zero.

 

The interaction thus far had required just over 82 seconds.  When I responded, “Subscriptions”, the system then said using an apparently different voice talent:

 

Thank you!  Transferring your call to: Wall Street Journal subscriptions and customer service.

 

After over 9 seconds of silence, I heard a telephone ring followed by:

 

Thank you for calling Dow Jones, publisher of the print and online editions of the Wall Street Journal and Barrons.

Our customer service office is closed at this time.  For tips on managing your print subscriptions on the web, or through our automated system, please press one.  For information on your subscription to W S J dot com, or Barrons dot com, please press two.

 

The system paused for approximately 2.0 seconds of silence and then repeated:

 

Thank you for calling Dow Jones, publisher of the print and online editions of the Wall Street Journal and Barrons.

Our customer service office is closed at this time.  For tips on managing your print subscriptions on the web, or through our automated system, please press one.  For information on your subscription to W S J dot com, or Barrons dot com, please press two.

Hearing nothing about subscription rates, I saw no point in continuing and I hung up.

 

The Diagnosis

1.      The system apparently does not support the task I intended to perform.

2.      Prompts are wordy and often confusing.

3.      Prompts and messages seem to be pointlessly repeated throughout the interaction.

4.      The System seems to mix DTMF-only modules in with speech modules.  This may be due to a transfer to another, independent system however.

5.      Overall production quality is below average.

6.      Production quality varies between the two voice talent prompt sets.

7.      Almost all system prompts are too long or unnecessary.

8.      At least one long silence was observed.

9.      While purporting to be a speech system, (i.e., a “speech enabled automated operator”) the system scarcely permits the user to speak.

10.  Users must hear numerous and drawn out preemptive prompts.

 

The Prescription

1.      Trim or remove any unnecessary prompts and remove all preemptive prompts.

2.      Create prompts that more clearly communicate user choices.

3.      Eliminate silences.

4.      Better integrate DTMF and speech modalities.

5.      Improve production quality and voice prompt recordings.

6.      Dramatically speed up the interactive pace.

 

The Prognosis

Average.

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

Sept 10 2007

Walt and Walter’s VUI View: Social Security Administration

The Caller’s View from Walt:

This system really makes it difficult to get to an agent.  It tells you to go to the web site, use the IVR, ignores you, hangs up on you, tells you to call back when it is less busy.  Much to verbose and often forces you to listen to long prompts.  Not at all consistent.   Sometimes T-T overstrike works and other times it is ignored. Prior to getting to an agent, I entered my SSN, DOB, Mother’s Maiden name, and place of birth.  The agent had none of this information and I had to provide it all again.  She pretended that it was a security issue, but finally admitted that she did not have any of the information that I had entered to the IVR.  The systems forces you to tell it that English is your language.  Tells you to go to the web site.  This is a long and tedious implementation that wastes the caller’s time.  Not at all quick-&-easy.

Dr. Walter Rolandi examination of the Social Security Administration customer support line
The Examination Session
The patient this month was a Social Security Administration IVR.  My examination task was to call the system to obtain the location of the nearest Social Security field office.  The system can be reached at 1-800-772-1213.  Only one call was required to accomplish the task although one more call was made to further explore the “main menu”.

Call Details:

The system answers the call using a female voice talent with a stereotypical greeting, followed by an awkward (almost 2.0 second) silence and some of the “usual IVR cliché suspects”:

Thank you for calling Social Security.

(silence)

To continue in English, press one now or say, English.

 

When I said, “English”, the system replied:

You’ve reached our automated services.  If you’d prefer to speak with an agent, please call this number between 7:00 AM and 7:00 PM, Monday through Friday.

Many services are also available on our Internet site at double-u double-u double-u dot Social Security dot G O V.

All this required just over 32 seconds.  The system then says:

 

Please tell me briefly the reason for your call.

As I opened my mouth to speak, the system continued:   

For example, you could say, “I need to change my address.”

I then said, “I want to find the address of the closest office”.  After over 3.0 seconds of silence, the system said:   

 

Sure.  I can help you find information about the Social Security field office for your location.

What’s your five-digit zip code?

 

I replied, “29209” and after another 3.0 seconds of silence, the system continued:

 

 

OK.  I found it.  You may want to write this down.

And if you don’t get it all the first time, you can hear it as many times as you need.

 

For zip code: 2 9 2 0 9….

 

At this point, another female voice begins to read 31 seconds of information about the nearest office.  These prompts were approximately two times louder than the prompts played up to this point.  After all of the information is read out, the less vocal voice talent returns to say:

 

That’s all of the information I have for that office.

Note that some offices don’t list their phone numbers or mailing address or their hours.

Should I repeat that, look up a different zip code, or, return to the main menu?

I said, “Main menu”.  After four seconds of silence, the system said:

 

Main Menu!

If I can help you with anything else today, just say briefly, what it is.

If you’re done, just hang up.

 

Having completed my task, I hung up.  I called the system again however, this time saying, “I want to get an address” at the main menu.  The system launched a 30-plus second information stream about “how to change your address”.  I could not make it stop or go back.  Finally, it asked me a yes-no question to which I replied, “Main menu” which, thankfully, enabled my escape from the “address change” ordeal.

 

The Diagnosis

1.      The system performed the initial task reasonably well but the second call was an IVR disaster.

2.      Several IVR clichés are present.

3.      Preemptive prompt content is present.

4.      Strange and awkward silences follow speech input.

5.      Production quality is lacking between the two voice talent prompt sets.

6.      The system is only slightly inappropriately anthropomorphic.

7.      Many prompts would benefit from trimming.

8.      Several prompts are unnecessary and should be eliminated.

9.      Injudicious use of barge-in prevents user control of the interaction.

 

The Prescription

1.      Eliminate IVR clichés.

2.      Trim or remove any unnecessary prompts and remove all preemptive prompts.

3.      Tune ASR end pointing to eliminate silences.

4.      Normalize all voice talent prompts to ensure constant loudness level.

5.      Rethink barge-in policy.

6.      Incorporate some sort of “cancel”, “stop” or “go back” function.

7.      Reduce time-to-task-completion requirement.

8.      Consider eliminating the SLM “main menu”.

 

The Prognosis

Reasonably good.

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

June 7 2007

Walt & Walter’s VUI View: on the Dell Customer Support Line
The Caller’s View from Walt
I received a brochure from Dell that was directed at small and medium businesses.  I called the number that was shown on the front page of the brochure (1-877-781-3355).  I encountered a greeting and menu which attempted to categorize my call by obtaining additional information from me.  It also asked me if I was agreeable to participating in a post-call survey.   After going through all of this, it then announced to me that they were closed and that I should call back during “normal” business hours.  The time was 8:30AM.  It hung up on me without providing the opportunity for me to leave a message or a callback phone number.   I followed up with other calls during “normal” business hours.  At the Dell web site, they list twelve (12) different numbers to call.  They categorize them by function (make a purchase, check order status, etc) or by who you are (small business, federal government, higher education, etc.),  This seems nice except that when you call any of these numbers you find that they seem to be something that someone dreamt up and forgot to implement.  CTI is missing.  This means that all of this categorization and information gathering is just wasting the caller’s time, since you start all over again with the agent that you finally reach.

Dr. Walter Rolandi examination of the Dell customer support line
The Examination Sessions
In today’s examination, my task was to call a Dell Computer toll free number and get information about buying a laptop computer.  The system can be reached at 1-877-781-DELL (1-877-781-3355).   While only one call was necessary to accomplish the task, four additional calls were made in order to explore other system features.  The successful “laptop computer information” call required 138 seconds to get to an appropriate sales agent.

Call Details:

When the system answers, the caller immediately hears an acceptably brief welcome message and an abbreviated “your call may be recorded” message.  The Main Menu immediately follows: 

To dial an extension, please press one.
To make a purchase, get pricing information or complete an online order, press two.
For technical support, or for software and technical questions, press three.
For all other issues related to a purchase already made, press four.
To repeat a menu at any time, press seven.

All this requires just over 22 seconds.   When I Pressed DTMF2, I heard:

Changes to orders or order status, press one.
Purchases for businesses with fewer than 400 employees, press two.
Home purchases, three.
For all others, four.

I pressed DTMF3 and then heard the following 29 second prompt set:

This voice activated system allows you to speak your response rather than pressing buttons on the phone.  Feel free to interrupt me at anytime when you know what to say.

To get you to the right sales agent please choose a category that best describes what you’d like to buy.  You can say the one you want anytime:

Computer systems
Ink and toner
TV
Electronics and Accessories 
Spare parts
Software or…
None of these.

I answered, “Computer systems” and after three seconds of silence I heard:

Are you interested in:

A desktop, Notebook or an XPS system?

Another three seconds of silence followed my answer, “Notebook” and then,

Thanks!
One last thing before I get you to an agent…
Would you like to see if you qualify for financing using our automated system?

After I said, “No”, I heard another 29-second message:

Ok.  I’ll connect you to the next available notebook computer sales agent.  One moment please.

As part of Dell’s continuing effort to improve your customer experience, we would like to invite you to participate in our random Customer Satisfaction Survey after you have spoken to one of our representatives.   Taking the survey will not effect your position in or call queue.  If you would like to participate in this survey, please press one now.  If you do not wish to participate, please press two now.

I spoke the word, “No” to which the system replied:

Thank you.  We will answer your call shortly. 

An appropriate sales associate answered after approximately 11 seconds of beeps, bops, phone rings and music on hold.

The Diagnosis

1.      The system mixes DTMF and speech modalities somewhat unpredictably.

2.      The system is not inappropriately anthropomorphic.

3.      No particular persona excesses were evident. 

4.      At least one menu contains too many options.

5.      While some prompt trimming is evident, most prompts are far too long.

6.      Many prompts are unnecessary and should be eliminated.

7.      All of my calls entailed the “random” invitation to take the Customer Satisfaction Survey.

8.      All of my calls entailed an invitation to apply for financing, even a call in which I inquired about “spare parts”.

9.      There is some evidence of non-standard DTMF usage. (DTMF7 for “repeat”).

10. Turn-taking pace in speech modality (3.0 second ASR response) seems slow.

The Prescription

1.      Universally support DTMF, speech of both.

2.      Adopt established industry best practices for the support of DTMF.

3.      Reorganize menus limiting their offerings to three items.

4.      Speed up turn taking.

5.      Eliminate “financing invitation” and “random survey invitation”.  Allow CSR to offer these options to the caller.

6.      Further trim all unnecessary prompt content.

Outpatient surgery would vastly improve the usability of this system, at least for callers seeking sales-type information.  Eliminating the “invitations” alone would probably cut 50 seconds off of this task.

The Prognosis

Very good indeed.

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

May 10 2007

Walt & Walter’s VUI View: on the Virgin Mobile Customer Support Line

The Caller’s View from Walt

This is the one of the few test/evaluations that I’ve been involved in where I am not an actual subscriber or customer.  The Virgin Mobile VUI (1-888-322-1122) has been touted as an example of a persona implementation that is targeted at a specific demographic and quite appropriate for that demographic.  The original developer (Nuance) has hyped the Virgin Mobile implementation on a number of occasions.  In fact, the Virgin Mobile VUI implementation was a finalist in the the Best Practices competition and received a “Best Premium Service” award from Nuance   Since text-messaging is so popular with the user community that Virgin Mobile targets, I was bit surprised that Virgin Mobile would even bother to offer a telephone speech system for their subscribers.  Why not provide a text-messaging support capability so that the users can use the communication mechanism that they prefer?  I asked the Virgin Mobile marketing people about this and they said that they were working on this and hoped to have this soon. The VUI is certainly most irritating and time wasting.  I’m not part of the Virgin Mobile target demographic, but neither were the folks that implemented it and crow about how appropriate it is.  The persona (Simone) that Virgin Mobile used was that of a young inner city girl with extremely poor language skills.  My teenage grandchildren don’t speak like this at all and I’ve not run into many other teenagers that do.  I would attribute the use of a voice such as Simones to nothing more than a disconnect between what Virgin Mobile marketing views as their target customer and what they are really like.  I simply can’t imagine that a reputable enterprise would even consider using Simone as a spokesperson that would present an appropriate public image.  Even if you find the voice amusing at first, I would doubt that it is still as amusing when you deal with Simone multiple times.  Aside from the Simone persona, this VUI implementation violated most of the gethuman standards.  In fact, the only gethuman standard that it passed was that it did provide DTMF backup to speech, which is what the vuids folks were objecting to.  Imagine that – if the Virgin Mobile implementers followed the guidance of the vuids persona folks, they would have been able to achieve a zero gethuman score.   The Virgin Mobile agents failed the “decent  communication ability” test, even though it seemed as though they were based in the U.S.A.   They seemed to have an attitude.  When they made you repeat information that you had previously provided to the IVR, no remorse was expressed.  The attitude seemed to be: “tough baby!”  This was even more infuriating since the VM IVR forced you provide information before it would connect you to the agent. Virgin Mobile should accelerate the implementation of their text-message customer support service and then shut Simone down.  The teenagers have a lot of reasons for using text-mesaging rather than voice. I would add the experience that they had with Simone, as another good reason to prefer text-messaging to speech.  

 

Dr. Walter Rolandi examination of the Virgin Mobile customer support line

The Examination Sessions

In today’s examination, my task was to call a Virgin Mobile toll free number and enquire about opening a new account.  The system can be reached at 1-888-322-1122.   The number is advertised on the company’s website on a “Contact Us” page (http://www.virginmobileusa.com/about/contact.do).   One example of the text that describes the number follows:

 

“If you have questions or comments about Virgin Mobile, or the services and products we offer, please contact our corporate office. Dial * 86 or 1-888-322-1122 from your Virgin Mobile phone if you need to reach us right away!”

 

Presumably one of the questions a person would call about is how to initiate a service contract or get information about a new account.   Be that as it may, it required two calls to the system to determine that it does not obviously support such enquiries.   The system apparently hung up on me after one timeout and two speech recognition errors on the first call.  It required 91 seconds.   On the second call, I was able to transfer to a CSR after 136 seconds of confused interaction with the IVR. 

 

First Call Details:

When the system answers, the caller hears approximately 3.5 seconds of silence followed by what appears to be a truncated musical audicon during which a man says, “Virgin Mobile at your service”.  Due to the volume of the accompanying music, I had to listen to the recording three times to determine what he was saying. 

 

A Spanish option follows approximately 2 seconds of silence:

For English, say English.  Or press one.

Para Español, diga Español.  O marque dos.

I said nothing and after 4.6 seconds of silence, the system announced:

Hey!  Wassup?  I’m Simone and this is Virgin Mobile at your service.

(.6 second pause)

Whadya wanna do?  

Say, Add money or top up or press one.

Get my account balance or press two.

Activate my phone or press three.

Change my offer or press four.

Or for other stuff say, More options or press five.

(.6 second pause)

You can tell me the answer just interrupt me as soon as you know what to say.

Not hearing anything that appeared to be related to opening an account, I was confused and said nothing.   After 8 seconds of silence, the system said:

Sorry.  What was that?

Say, Add money or top up or press one.

Get my account balance or press two.

Activate my phone or press three.

Change my offer or press four.

Or for other stuff say, More options or press five.

Still confused, I decided to venture a guess, hoping that new account information might be under “More options”.  I said, “Press five”.  After 2.5 seconds of silence, the system responded,

Sorry, I’m still not getting that.

If you wanna add money or top up your account press one.

To hear your account…

I barged in saying, “Five’.  After approximately 2 seconds of silence, the system then said,

Thanks for calling Virgin Mobile.  Talk to you later.

Not sure what had just happened, I waited through 8 more seconds of silence at which time a busy signal became audible.  The system had apparently hung up on me.

 

Second Call Details:

Again, the system answered the call with 3.5 seconds of silence followed by the truncated musical audicon.  Wanting to speed things up and quickly navigate to the “More options” path, I entered DTMF1 (for English) during the audicon.  The system ignored the input.  As the system then said,

For English,

I entered DTMF1 again.   The system responded, suppressing the rest of the Spanish option prompts.  After approximately 3 seconds of silence, it then began the main menu,

Hey!  Wassup?  I’m Simone and this is Virgin Mobile at your service.

(.6 second pause)

Whadya wanna do?  

Say, Add money or top up or press one.

Get my account balance or press two.

Activate my phone or press three.

Change my offer or press four.

Or for other stuff say, More options or press five.

(.6 second pause)

You can tell me the answer just interrupt me as soon as you know what to say.

Before I could enunciate “More options”, the system said,

Sorry.  What was that?

Say, Add money or top up…

It had apparently responded to background noise.   I entered DTMF5 to dial-through the rest of the menu.   After 2 seconds of silence, the system responded,

Here’s more.

You can say, Transfer a phone number or press one.

Buy a new phone or press two.

Hear about Virgin XL or press three.

Ask a question or press four.

Or say, Start over or press five.

Which did you want to do?

Still not hearing anything that was obviously related to opening an account, I responded, “Start over”.   After 3 seconds of silence, the system repeated the now familiar and completely stale menu:

Whadya wanna do?  

Say, Add money or top up or press one.

Get my account balance or press two.

Activate my phone or press three.

Change my offer or press four...

I barged in saying, “Change my offer”, thinking that it might lead to new account information.  After 3 seconds of silence, the system then said,

So, what’s your Virgin Mobile phone number?

Just press or say the number starting with the area code or say, I don’t have one.

Given that my intention was to find out how to obtain a Virgin Mobile phone number, I said, “I don’t have one”.   Another 3 seconds of silence followed after which the system said,

Ok, so I can get your call to the right live advisor, tell me why you’re calling today.

Say, Returns or exchanges or press one.

Activate your phone or press two.

Problems making or receiving calls or press three.

Disconnect service or press four.

Or say, It’s something else or press five.

When I said, “It’s something else”, the system then said,

Let me get you to a live advisor, hold on.

They might ask you to repeat some of the information you gave me, OK?

Oh and just so you know, your call may be recorded for quality assurance.

I then heard two loud beeps followed by 7 seconds of loud music.  A very pleasant CSR (Christina) soon answered and to whom I explained what I had been trying to do.  I asked her if new account information was available through the system she answered, “yes” indicating that I should have taken the “Activate your phone” path.   I pointed out that the choice was far from obvious and she agreed and apologized.  She went on to explain that in order to get new account information, one must talk with a live agent and choosing the “Activate your phone” option would take the caller to an agent.

 

The Diagnosis

1.      The system is indeed gravely ill.

2.      This appears to be a case where presumed entertainment value came before simple and basic functionality in design.

3.      The system is egregiously anthropomorphic.

4.      It has one of the most inappropriate personas I have ever examined.

5.      The menu choices are couched in slang (“add money”, “top up”, “change my offer”)

6.      Menu choices are unclear.  Are “add money” and “top up” the same or different things?

7.      Menu choices illogically include “or press <N>” construction.

8.      Menu grammars do not permit users to speak the DTMF numbers.

9.      Some menus contain too many options.

10. Apparently the system disconnects the call if three errors occur.

11. Some prompts contain preemptive instructions that pointlessly tell the user how to respond.

12. Essentially all system prompts are too long, unnecessary or stylistic to no practical end.

13. Overall system organization is questionable.

14. “Change my offer” path leads to curious consequences.

15. The system makes it difficult to get to a human.

16. Turn-taking pace is sluggish.  ASR response seems slow.

17. There are occasional unexplained silences.

18. It has an annoying and unnecessarily complicated Spanish option.

19. It contains IVR clichés (“…your call may be recorded for quality assurance”).

20. Production quality is at times questionable.

 

The Prescription

1.      Eliminate the persona and all anthropomorphic content.

2.      Reorganize menus limiting their offerings to three items.

3.      Eliminate all slang.  Describe menu choices in clear, concise and businesslike terms.

4.      Eliminate the Spanish option.  Support a different 800 line for Spanish.

5.      Adopt established industry best practices for the support of DTMF.

6.      Devise a less punitive strategy for dealing with errors.

7.      Speed up turn taking.

8.      Eliminate all preemptive prompts and all unnecessary prompt content.

9.      Eliminate trendy audicon and loud pre-transfer music.

10. Eliminate or trim the ‘call recorded for quality assurance’ cliché prompt down to some minimalistic alternative that serves the same (loathsome) legal purpose.

11. Determine cause of and eliminate unexplained silences.

Only the most drastic interventions could save this system but it is not beyond hope.  With major surgery, and in particular, a lobotomy, the patient might dramatically improve.

 

The Prognosis

The prognosis would be good if only Virgin Mobile would realize how badly treatment is needed.

 

 

 

 

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

Apr 9 2007

Walt & Walter’s VUI View: on the Verizon Broadband Support Lines

The Caller’s View from Walt

We had some severe power outages this month that were caused by a faulty local transformer.  In replacing the transformer, the power company had difficulty finding a good ground and managed to fry my DSL board that is provided by Verizon.  This seemed like an opportune time to explore converting to the Verizon Fiber Option broadband offering that Verizon is promoting.  I called the 800-567-6789 number that Verizon listed for broadband sales and service inquiries. What a painfully slow system this is !  Huge delays after every response.  Limited vocabulary but it still gave me the I’m sorry, I didn’t understand that” a number of times.  Long and useless prompts.  What value in telling the caller that they are at the Main menu ?  Had me validate my phone number two times and when I finally got to the agent, he did not have it and I had to provide it a third time.   The system announced that it was going to check if the FiOS service was available at my address. It finally came back and announced that it was transferring me to an agent.  I finally reached an agent that did not have a clue what I was calling about and started all over again to check if FiOS was available at my address.  He finally determined that it was not available but was not able to tell me when it would be available at my address.  He announced that it has been available in my town for over a year, but not at my address.   It took 163 seconds for the system to figure out what I was calling about.  A human could have done this in 10 seconds, tops.  Unnecessarily wasting over two minutes of the caller’s time is terrible.  Worse – to treat a caller that wants to buy something this way is really bad business.   I would have called Comcast to switch to their broadband service, but, unfortunately, they are just as bad.  I guess that when you are a monopoly for a hundred years, sales & marketing skills just are not needed and developing them is virtually impossible.  After resigning myself to being stuck with Verizon DSL for the foreseeable future, I called the technical support line to get assistance setting up my new modem.  This apparently is outsourced offshore and I had a difficult time understanding the technician that had a heavy accent.  The only positive thing was that he did not say Great ! after every phrase that he uttered.

 

Dr. Walter Rolandi examination of the Verizon Broadband customer support line

The Examination Sessions

A Verizon Online Internet Services IVR was the patient today.  It can be reached at 1-800-567-6789.  My task was to call the system and obtain information about Verizon FiOS, Verizon’s Fiber Optic broadband service.   In order to get information about the service, one must provide a telephone number that is in an area where the service is offered.  A friend of Walt Tetschner (and Verizon customer) who lives in Virginia volunteered his number for the examination.

 

To get information about Verizon FiOS, one must first locate the topic in the system’s tree and then be transferred to a CSR who can help.   Only one call was required to complete the task.  It took approximately 163 seconds to get to the point of transfer.

 

Call Details:

ed some sort of musical stimulus which lasted 0.44 seconds and then about 1.5 more seconds of silence.   Finally the system says:

 

Starting with the area code please say or enter your account phone number now or say ‘I don’t know it’.  If you’re interested in ordering new service, say, ‘New service’.

 

I responded, providing our volunteered phone number.   After 5 seconds of silence, the system said:

 

Lemme make sure I got that right.

804-NNN-NNNN

Is that correct?

 

I answered, “Yes” and after 11.5 seconds of silence, the system again played the 0.44 musical stimulus and then announced:

 

You’re at the Main Menu.  To start over at any time, say Main Menu.  (ding audicon)

Which are you calling about?  Tech support, your account or orders?

 

Confused, I said nothing.  After 5 seconds of silence, a timeout message that elaborated on the “tech support, your account or orders” options was read.   It included:

 

…to place an order or to check the status of an order, say ‘orders’.

 

Appearing to be my best guess, I said, ‘Orders?’  After about 3.5 seconds of silence, the system said,

 

All right, orders.

If you’re calling about an existing order, say, ‘status’. 

Or, to place a new order, say, ‘new’.

 

Again, I ventured a guess and replied, ‘New?’  After a full 5 seconds of silence, the system asked,

 

Which kind of new service are you interested in?

Service or equipment?

 

Once again mystified, I guessed, ‘Service?’  After 3 seconds of silence, the system asked,

 

So, which service would you like to order? 

DSL, Verizon FiOS or other?

 

Greatly relieved, I replied, ‘FiOS” and after 3 seconds of silence, the system explained that it needed to confirm that the Verizon FiOS service was available in the area of interest.  It read back the telephone I had previously provided and asked if it was the number for which I wanted service.  I answered, ‘yes’.  After 3 seconds of silence, the system announced that it was checking to see if service was available “for that number”.  After 5 seconds of silence, it returned saying that it had checked the line and that the call was being transferred to a customer service agent.

 

The sequence ended with the familiar:

 

By the way, your call may be monitored or recorded to assure quality customer service.

after which the call was transferred.

 

The Diagnosis

1.      While evincing some of the industry’s best practices, the system is nevertheless quite ill.

2.      Finding the FiOS information requires either uncanny luck or near clairvoyant powers of perception.

3.      Almost a third (50 of 163 seconds) of the time needed to stumble toward task completion was spent listening to ‘dead air’.

4.      The menu choices afforded by the system do not naturally seem to suggest what follows.

5.      The system is somewhat anthropomorphic.

6.      Some prompts contain preemptive instructions that pointlessly tell the user how to respond.

7.      Some system prompts are too long or unnecessary.

8.      Use of audicons appeared to be non-systematic.

 

The Prescription

1.      Determine the cause of unexplained silences and reduce or eliminate them.

2.      Tone down artificial attempts to simulate human conversational interaction.

3.      Support a different 800 line for Spanish.

4.      Infuse menus with more “clues” as to where they might lead.

5.      Trim prompts of preemptive and unnecessary content.

6.      Devise a more coherent and informative strategy for the use of audicons or eliminate them.

7.      Trim the ‘call recorded for quality assurance’ cliché prompt down to some minimalistic alternative that serves the same (loathsome) legal purpose.

 

Ironically, this system is better designed than many speech IVRs.  Yet, performing the ostensibly simple task of obtaining FiOS information required a series of lucky guesses.   The pervasive, unpredictable and unexplained silences make using the otherwise acceptably designed system an awkward experience.  If the cause of the silences could be overcome, menus clarified and prompts trimmed and or rewritten, things could get a lot better.

 

The Prognosis

Reasonably good. 

 

 

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com

 

 

Mar 8 2007

Walt & Walter’s VUI View: on the Cingular Service Support Lines

The Caller’s View from Walt

This month we called the Cingular Customer Service Lines. Since Cingular is one of the largest BeVocal customers, we thought it was timely to take a peek at how well they have implemented their customer support line. I’m not a Cingular customer so I could not access the Cingular Customer Support service.  I could see about becoming a Cingular Wireless customer, though.  On the Cingular web site, I located the following:

Considering Cingular?

Call us for more information about coverage, offers, cool phones and anything else you need to know.

1-888-333-6651

This looked like the appropriate number for a potential customer to call, obtain additional information and become a Cingular Wireless subscriber.  So I next called this number.

Thank you for calling Cingular Wireless – now the new AT&T.

For English press 1

(My response: 1)

For assistance with your current account, press 1

For status on orders placed on Cingular.com, press 2

If you would like to purchase new service over the phone or add a line, press 3

(My response: 3)

For assistance with your existing account, press 1

For assistance upgrading your existing phone, press 2

To purchase new service, or add a line, press 3

(My response: 3)

Thank you for calling Cingular Wireless - now The New AT&T.  Presently our center is closed. Please try your call again later. 

These folks are as incompetent at sales as it gets.  Firstly, they seem to be having trouble figuring out what their name should be.  Lets see now: when Cingular acquired AT&T Wireless a few years ago, they got rid of the AT&T name and replaced it with Cingular.  Now AT&T is acquiring Cingular and has decided to go get rid of the Cingular name and replace it with AT&T.  Or is it The New AT&T?  These folks should just pick a name that they like, tell the world what it is, and get on with it.  Stop the babbling about the name.  No one cares but the marketing people at Cingular.  I wonder what happened to all of the AT&T signs that they took down from the stores a couple of years ago when they changed the name from AT&T wireless to Cingular Wireless? If you look at the menu that they dumped on me, rather than responding to the menu item that I selected, it came back at me with a menu that repeated most of what was in the first menu.  It dropped the order status item and referred to the account as “existing” rather than “current”, but otherwise was the same as the 1st menu.  When I again responded with a request to “purchase new service or add a line”, they announced their name change again, told me that they were closed and that I should call back when they were open.  What is wrong with these people?  I’m a new potential customer and they don’t  even bother to capture my callerID or let me leave a voice mail.  Their marketing people spend millions of $s trying to talk to me and when I want to talk to them, their call center tells me to go away.  Wasting my time dealing with all of these menu choices and then telling me that they were closed is about as offensive as you can get.

 

The system that Cingular has to support their existing subscribers is different than the system for dealing with new customers.  The voice for the prospective customer is that of a mature female while the voice for the system that supports existing customers sounds like that of a an adolescent female.  One system asks the 95% of the callers that are English-speaking to select the preferred language while the other one requests the Spanish-speaking callers to do the selection.  The line for potential customers also accommodates existing subscribers that called this line in error.  They are rewarded for calling the wrong number by being offered a menu which directs them to the proper customer support line.   The downside is that you are wasting the time of a potential Cingular customer and working hard to alienate him/her.  If you are a subscriber that has called this number in error, when you select the menu choice to inquire about your existing (or current) account, you are transferred to the customer support system that my colleague has analyzed below.  Of course, the caller is subjected to the Cingular-going-to- AT&T greeting all over again.  They also decide to play the “this call may be recorded for quality purposes” announcement which is inappropriate.  If  the Cingular subscriber decides to check the status of an order that they had placed with www.cingular.com, they are routed to a speech-enabled service.  This service does not announce the name change from Cingular Wireless to The New AT&T.  No one told them?  Didn’t attempt to analyze this system in any detail but noticed that it was not able to deal with a natural number when I spoke one.  Error response was: “I heard 01720, is that correct?’ 

 

Dr. Walter Rolandi examination of the Cingular customer support line

The Examination Sessions

Today we examined a Cingular Wireless customer support line.  It can be reached at 800-331-0500.   My task was to call the system and determine how many minutes had been used to date.   I made several calls to explore the system although only one call was necessary to complete my task.

Call Details:

When the DTMF-only system answers, the caller hears a brief greeting which includes a marketing plug for “the new AT&T”, followed by a Spanish option which is available via the “star” key.  

 

Thank you for calling Cingular Wireless, now the new AT&T.

Oprima el asterisco para informacion en Espanol.

 

(2-second pause)

 

Please enter the wireless phone number you are calling about starting with the area code.  For information on Cingular products and services or for help with a recent order, press one for sales.

 

(entered phone number)

 

Please hold while we access your account information.

 

(4-second pause)

 

Main Menu:

To pay your bill or get other bill related options, press one.

To check your minutes, press two.

To get help with voicemail, press three.

To report a lost or stolen phone, press four.

For sales, press five.

To get help with issues not covered by our automated options and to speak with a customer service professional, press zero.

To repeat these options, press star.

 

The Main Menu was read in approximately 22 seconds.  When I then pressed DTMF 2, the system asked me to enter the billing zip code for “security”.   What immediately followed was a barrage of information about various kinds of minute categories, how much had been used and how much remained.  It lasted for 66 seconds.  I completed the entire task in 131 seconds.

The Diagnosis

1.      This is a straightforward and reasonably healthy system.

2.      It is fairly benign in its support of DTMF dial through.

3.      The system is however afflicted with a few annoying and superficial illnesses, similar in analogy to “rashes”.

4.      It proffers a “sales” opportunity immediately upon prompting for the customer’s cell phone number.

5.      It stupidly responds to user error with a prompt that laments, “we did not recognize your response”.

6.      Many system prompts are too long or unnecessary.

7.      System seems to try to compensate for long prompts by reading them out very quickly.

It is possible to get to a CSR quickly although Cingular does seem to try to identify the customer account first.   The system is easy enough to use and supports fairly rapid task completion.

The Prescription

1.      Replace pointless DTMF error response prompts with something helpful.

2.      Support a private line for Spanish speakers.

3.      Trim all prompts and eliminate initial sales pitch.

4.      Slow down prompt presentation.

This system could be put into tip-top shape by simply following the above prescription.  

The Prognosis

Excellent. 

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 


Feb 5 2007

Walt & Walter’s VUI View: on the Apple Service Support Line

The Caller’s View from Walt

This month we called the Apple Service Support line. I was greeted with a 4 item T-T menu that included a technical support item and a selection that would let me locate an authorized Apple dealer in my geography.  I tried the support item which was the first one on the menu.  Got transferred to another system which was speech enabled.  When I was asked what Apple product I was calling about I said “iPod Nano”.  I was a bit surprised that it responded by saying: “I think you said iPod Nano.  Is this correct?  Please say yes or no”.  I thought that I had said it clearly.  I hung up and tried again.  It responded in the same fashion.  What is wrong with these people?  Why are they wasting my time like this?  They seem to be going through an irritating and time wasting confirmation process for everything that is said by the caller.  After dealing with the awful speech recognition experience, I connected to a human.  Getting to the human was a difficult process.  It first refused to recognize my agent request upfront and dumped me to the speech-enabled self-service.  I had to go two-out-of-three with it before it reluctantly transferred me to a human.  The human did not have any of the information about my experience with the speech recognition. Next, I thought that I would try to locate an Apple dealer in my geography.  After selecting the dealer locator option, I received the “this call may be recorded message” which was inappropriate since it was not transferring me to an agent where the call was likely to be recorded.  I was then transferred to still another system (a T-T one where I encountered this menu that went on-&-on and kept telling me to go to the Apple web site (and since it must have figured that I was too stupid to figure it out, that it had to tell me over-&-over again that it was www.apple.com )   It had option after option that it dumped on me that had nothing to do with finding a dealer in my geography and for each option I was instructed to go to the Apple web site.  These folks just don’t relate to the caller at all.  I’m well aware that Apple has a web site and don’t really need instructions on how to find it.  Why they somehow think it is appropriate and effective to keep on telling me over-&-over that I should go to their web site escapes me.  After listening to this long-winded menu, I selected the one that was for locating a dealer in my geography.  It then asked me if it was for business or personal use.  I selected business.  It then transferred me back to the original system that announced that they were closed and that I should call back during regular business hours and hung up on me without capturing a messaging or even my phone number.  Wow!  These guys really know how to irritate a caller.  They tell me that I can find an authorized dealer in my geography by pressing 2 and then waste my time with all sorts of options and then finally tell me to call back when they are open for business.  After hours, a request to connect to a human is responded to by telling the caller to call back during normal business hours (6AM to 6PM PT).  The fact that the Apple support line does not capture my phone number is pretty awful.  All of the repetitive menus are pretty bad the first time that you hear them but the impact is much worse when you call again and they dump all of this stuff at you again.  The Apple support service is made up of a number of different applications that sound and act different from each other.  They will repeat information that another application has already provided and occasionally even disagree and provide different information.  They use the # key for Repeat. It should be pretty apparent why the Apple iPhone is going to be a huge failure.  Apple knows nothing about telephony and it shows pretty badly on their telephone support line.

Dr. Walter Rolandi examination of the Apple customer support line

The Examination Sessions

An Apple technical support line was the patient today.  I called it at 800-275-2273.   My task was to call the system and request technical support for an iPod Nano.   Only one call was necessary but I made three or four more just to explore the options.

Call Details:

For technical support press one

Status on a previously scheduled repair, two

  To extend service or support beyond Apple’s complementary period or for questions about the Apple Care Protection Plan, three

To find an authorized Apple reseller or service provider, four

On one call I pressed DTMF6 just to see what would happen.  The system quickly responded, “Your entry was not understood.”  Pressing DTMF1 transfers the caller to a speech recognition system that asks the caller to say the name of the product about which he is calling.  The system affords the caller some examples:

For example, you could say, Mac Pro, iPod Nano or Aperture.

 At this juncture, on three separate occasions, I clearly said, “iPod Nano” to which the system invariably replied::

 

I think you said, iPod Nano.  Is that correct?

Upon confirmation, the system asked one other question and then transferred my call to a customer support representative.  This process required approximately 70 seconds.

 

On another call, I pressed DTMF4 to locate an Apple dealer.  I was quickly transferred to another DTMF system that exemplifies many of the most common maladies in IVR.  The opening line of its 107-second Main Menu was that old IVR chestnut,

Please listen carefully to the following menu as the options have changed.

This was immediately followed by a web ad.  Then came menu item 1, menu item 2 which included 4 (four!) more web ads and instructions to call two 800 numbers for technical support, one of which was 800-275-2273, (the number I had called to reach this system), and then menu item 3, which in turn contained yet another web ad.

The Diagnosis

1.   Initial interaction is in DTMF only.  System may route a call to other systems that support speech or DTMF only.

2.   Opening Main Menu is succinct and nicely trimmed.

3.    Saying, “Your entry was not understood” in response to user errors is pointless and almost punitive.

4.      Speech recognition component seems unintelligently designed and poorly tuned.

5.      Dealer locator subsystem set a new record for web ad content (six!).

6.      Dealer locator subsystem prevents dial-through during opening prompt cliché and initial web ad, thus forcing users to listen.

7.      Overall production quality is questionable, being a hodgepodge of systems and subsystems.

Other than appearing unintelligent in its confirmation prompting, the system process, at least for callers merely seeking technical support of a product, actually is quite reasonable.   Some of the other subsystems that were sampled however are horribly ill.  In fact, this is the worst case of “web ad-itus” I have ever encountered.

The Prescription

1.      Eliminate pointless DTMF error response prompts.

2.      Tune speech recognition.

3.      Rethink the speech confirmation process.

4.      Eliminate all unnecessary prompt and prompt content.

5.      Get rid of all web ads.

6.      Permit users to dial-through universally.

This system could be restored to better health relatively easily with mere cosmetic changes to a number of prompts.  More invasive interventions may be necessary to improve the overall production quality of the system.  To the owners of this system I would prescribe at least an Apple-call a day.   I am sure they would perceive the system’s annoyances if they would only take a critical look.

The Prognosis

Fairly good. 

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

 

 

 

Jan 6 20077

Walt & Walter’s VUI View: on the Federal Trade Commission (FTC) customer support Telephone Self-service Line

The Caller’s View from Walt

This month we called the FTC customer support line.  The FTC is an independent agency that has the authority to adopt trade regulation rules that define unfair or deceptive acts in particular industries. Trade regulation rules have the force of law. The FTC reports to Congress on its actions.

When I called 877-382-4357, I was greeted by a monologue that just went on-&-on.  In addition to rattling off menu choice after menu choice, it managed to violate most of  the gethuman Core Principles including: wasting my time telling me about the Spanish option; telling me to go to their web site and that it was www.ftc.gov (surprise);  telling me that their menu had changed and that I should pay careful attention.  Thankfully, they did not tell me that "your call is important to us".  I guess that they could not get someone to keep a straight face long enough to record this one.  The 2nd time that I made an error, it hung up on me.  No voice mail or callerID capture. If you want to talk to us, call us back when we are available. When I called back during "normal working hours" the next morning. the only difference that I was able to notice was that when I requested that I be connected to a human up-front, it put me into a menu that got me to tell them if this was my first inquiry and to provide my zip code if it was, rather than hang up on me.  It still did not recognize my 1st request for a human and forced me to request it twice.  It announced a wait time of "approximately 4 seconds" (which took about 10 seconds to announce). In the wait announcement, it did manage to assure me that “your call is important to us’. I was then connected to a human in about 2 seconds.  His voice was clearly that of an American male and I made the assumption that the FTC was not outsourcing it's customer support to Mumbai.   He did not have my zipcode and had me provide it again. I did use the FTC speech-enabled system to order some of the FTC telemarketing documents.  It again droned on again about the FTC web site in gory detail.  When will this thing ever stop?  Capturing my mailing address was a painful process.  It also made me provide my phone number, although it wasn’t apparent why this was needed.  After I completed this order, I called again to order some additional FTC literature.  It did not have my address and made me enter it all over again.

 

Dr. Walter Rolandi examination of the FTC customer support line

The Examination Sessions

An FTC IVR was examined this month.  It can be reached at 877-382-4357.   The system offers seven (7) options on its Main Menu.   My task was to call the system and order one of the “more than 400 publications available from the FTC”.   Only one call was necessary.

Call Details:

The call is answered with a brief greeting followed by a Spanish language option.  A 7-item Main Menu (of sorts) immediately follows.  There is actually a 5-item Main Menu but it is preceded by two other options one to “press 6” and another to “press 8).   The menu items are extraordinarily long.   The publication ordering option was Main Menu 5, or the seventh of seven total options offered.  When the menu completed, I entered DTMF 5. 

           

The call required approximately 78 seconds to arrive at this point.  My choice was immediately followed by a 27-plus second prompt encouraging me to order the publications by using the FTC’s website.  This “web ad” was apparently intended to discourage ordering publications using the FTC IVR.   After reading the lengthy prompt, the system requires the user to again indicate his desire (by entering DTMF 1) to order a publication by mail using the system.  When I did so, the system failed to interpret the DTMF 1 signal and proceeded to repeat the entire 27-plus second web ad.

When prompted to do so again, I entered DTMF 1 and held down the key for approximately 1 second.   I was immediately transferred to another system, this one using speech recognition technologies, to complete the task.  What followed was a 5 minute and 38 second ordeal of name and address acquisition and confirmation.

 

The Diagnosis

1.      System is unbelievably stupid in its organization and prompting.

2.      Initial interaction is in DTMF only.  Subsequent interaction supports ASR.

3.      Main Menu is too long and illogically organized.

4.      System is complicated and tries to automate tasks that are inappropriate for automation.

5.      System failed to interpret confirmed DTMF input.

6.      System requires far too much time and energy to complete a simple task.

7.      The system’s value proposition is extremely questionable.

8.      System terminated the call upon task completion thereby not permitting me to do anything else without recalling the system.

9.      Production quality is below average.

10.  Most prompts could be eliminated, the rest trimmed.

Once again, we have a profoundly unhealthy system.   More than merely being ill, the system appears to be mentally ill as well.

 

The Prescription

1.      Determine why DTMF input could not be processed.

2.      Completely redesign the system eliminating most of its features.

3.      Employ well established best practices for prompting and error handling.

4.      Either adopt a design philosophy that systems exist to assist their users or don’t’ bother with 1-3.

This is a tragic example of “your tax dollars at work”.   Consider the absurdity of offering “more than 400 publications available from the FTC”.   The system is itself symbolic of the government bureaucracy that created it: most of it is utterly unnecessary and unwanted.  

 

The Prognosis

Not good.  Only the most extraordinary interventions should be contemplated and even radical surgery affords limited hope. 

 

Contact: Walter Rolandi, Ph.D.,
The Voice User Interface Company, LLC
+1. 803.252.9995,
wrolandi@wrolandi.com www.wrolandi.com

 

 

December 6, 2006

Walt & Walter's VUI View: on Eureka's customer support Telephone Self-service Line

The Caller's View from Walt

Eureka Company is also known as Electrolux Home Care Products North America. Electrolux is a huge Swedish-based company that holds a 28% share of the worldwide household appliances market. A slogan that Electrolux originally used for marketing their vacuum cleaners in the UK was: "Nothing sucks like an Electrolux." We tested the Eureka telephone support line (800-282-2886). This looked like it was something out of the past. Even someone like myself that is afflicted with retrophilia, had difficulty in finding much good about this implementation. The conclusion of my colleague (Walter Rolandi) was that "The Eureka telephone support line really sucks". I fully concur with his observation. I found this to be among the worst telephone self-service implementations that I've ever run into. The challenge of getting to a human is huge. If you press the right sequence of keys at precisely the right time, the Eureka system will begrudgingly attempt to connect you to a human. Otherwise, it responds to the caller's "errors" by abruptly hanging up. Even if you do everything just the way that they require, you still are not assured of getting to a human. I received a message: "Due to circumstances beyond out control, we are temporarily unable to take your call. Please call back at a later time." It then hangs up without offering voice mail or capturing your callerID. This went on all day. The Eureka line is both a customer support line and a sales line. If you indicate that you are interested in buying a Eureka product, the system tells you to call another number and hangs up. Imagine that ? a live customer is on the line that wants to buy something and the Eureka response is to tell them to hang up and call another number. Forcing the caller to listen to a long verbose menu of choices and not permitting overstrike is attrocious. A very primitive speech-enabled capability exists. It appears to recognize a caller speaking 8 different digits. Anything else it treats as a caller error. A Touch-Tone capability exists but you have to choose one or the other ? either speech or Touch-Tone. Of the ten gethuman standards, the only one that they did not appear to violate was the language selection one. It was so difficult to get to a human that it was difficult to determine if the agent spoke clearly. I'd suspect that for the few callers that actually manage to get to a human are so dazed by the experience that they don't even notice if the human speaks clearly or not. The Eureka telephone self-service looks like something that was implemented in the early days of telephone speech (circa 1986). Digits only and no overstrike were limitations of that era. Basic utility was for persons that only had a rotary phone. In fact, if you indicated that you had a Touch-Tone phone, then the system forced you to use only Touch-Tone. I give them credit for not telling callers to go to their web site (they problably did not have a web site when this was implemented). They also don't instruct the caller to: "pay attention since the menu has changed", and it certainly was not an overly lively persona. Amazingly, they did have the nerve to tell the caller that their call was important to them. When they upgrade, this will give them an opportunity to add all of the latest "torture-the-caller" techniques. This is an instance in which the gethuman evaluation scheme does not do proper justice to just how badly implemented the Eureka telephone self-service is. To compensate for this, we decided to award them an honorary gethuman score of 0.

 

Dr. Walter Rolandi examination of Eureka's customer service linee

The Examination Sessions

This month the patient was a Eureka speech enabled IVR which is available at 1- 800-282-2886. The system offers seven (7) options on its Main Menu. My task was to call the system and order a catalog. Two calls were necessary however due to the system's modal organization. It is organized in such a way as to encourage callers having DTMF to use DTMF. Using speech is an option made available only to callers who do not indicate they have touchtone phones. Thus, two calls were needed to traverse both paths of the tree.

Call One:

The call is answered with an 8.5 second greeting after which the caller is asked if he is calling from a touchtone phone. If the caller indicates that he is, he then hears a 7-item Main Menu followed by an additional (8th) option to repeat the information:

Thank you for calling the Eureka Company. If you have a touchtone phone, please press one now. Otherwise, Please stay on the line.

(User enters DTMF 1)

For information on warranty and where to obtain service, press one.

To purchase parts or accessories, press two.

To obtain an owner's manual, or, for information on how to replace your belt, press three.

For information on where to buy Eureka products, to receive product literature, become a Eureka dealer or for presales information, press four.

If you are currently a Eureka dealer or warranty station, press five.

For information on the style of bag and/or belt your vacuum requires, press six.

For all other information, press seven.

To repeat these options, press nine.

 

The call required approximately 55 seconds to arrive at this point. Approximately 3 seconds later, before I had had a chance to respond, the system said,

To avoid being disconnected, please make a selection.

For information on where to buy Eureka household vacuums, press one now.

For information on where to buy Eureka built-in vacuums, press two now.

To receive a Eureka product catalog, press three now.

For questions regarding Sanitaire household products, Sanitaire commercial products or to become a dealer, press four now.

For presales information, press five now.

(User enters DTMF 3)

Please enter your five-digit postal zip code now.

(User enters five-digit zip code using DTMF)

At the tone, please say then spell your first and last names.

(User says name)

Please verify that your name is: (system plays back recorded name)

If this is correct, press one now. If this is incorrect, press two now.

(User enters DTMF 1)

At the tone, please say then spell your street address and then say your city and state.

(User complies)

Please verify that your address is: (system plays back recorded address)

If this is correct, press one now. If this is incorrect, press two now.

(User enters DTMF 1)

Please allow seven business days for delivery. Thank you for calling Eureka.

(System hangs up after 8-second silence)

This entire process required 186 seconds.

 

Call Two:

The second call was basically the same as the first call except that I intentionally did not respond when prompted to use DTMF. After about a 6-second silence, the system read the Main Menu above only slightly modified to prompt user speech.

For information on warranty and where to obtain service, say one.

To purchase parts or accessories, say two.

To obtain an owner's manual, or, for information on how to replace your belt, say three.

For information on where to buy Eureka products, to receive product literature, become a Eureka dealer or for presales information, say four.

If you are currently a Eureka dealer or warranty station, say five.

For information on the style of bag and/or belt your vacuum requires, say six.

For all other information, say seven.

To repeat these options, say nine.

I wanted to see if DTMF was supported in the speech recognition mode. Upon hearing the entire alternative Main Menu, I entered DTMF 4 to which the system immediately responded:

Thank you for calling Eureka.

(System hung up after 8-second silence)

The Diagnosis

  1. System inexplicably segregates DTMF and ASR modes.
  2. Main Menu is far too long.
  3. System attempts to automate far too many tasks.
  4. Main Menu and sub-menu are complicated and confusing.
  5. System requires too much time to complete this simple task.
  6. System does not provide enough time to respond.
  7. System ominously threatens the caller with disconnection if they do not promptly respond.
  8. System is horribly unforgiving. It hung up on me in speech mode presumably for making one error.
  9. System terminated my first call upon task completion thereby requiring users to call back if they sought to perform more than one task.
  10. Production quality is questionable.
  11. Almost all prompts could be trimmed.

This patient is profoundly ill. Eight choices on a menu are far too many but if one considers the "ands and ors" contained on the Main Menu, it actually contains sixteen (16) choices. This is cognitively indigestible by most human beings. Overall, the system tries to do too many things and it is therefore big and complicated. Its speech interface is exceptionally primitive, apparently employing only a 10-digit grammar. This suggests antiquated technology which might perhaps explain the segregation of the DTMF and speech modes. Whatever the explanation, the system's modal organization is senseless.

The Prescription

  1. Completely redesign the system eliminating all but the most essential features.
  2. Employ well established best practices for prompting, timing and error handling.
  3. Permit dial through and barge-in.
  4. Use professional voice talent.

Traditional treatments and cosmetic changes would do little to improve the health of this system. This is a case for euthanasia.

The Prognosis

Without a complete reorganization, very grim indeed.

wrolandi@wrolandi.comm www.wrolandi.com


November 7, 2006

The Caller's View from Waltt

I called the 800-GO-FEDEX number.  This implementation was the best that we've tested so far. The gethuman VUI Standard score was 6 out of 10 which positions it 2 points higher than the best that we had previously seen. Used my CallerID to determine my location and find the nearest drop-off points. Having DTMF available but hiding it from the caller seemed like a strange thing to do. Never was able to figure out how to get it to repeat. It did not like *. Most of the calls to this line are from repeat callers. Keeping track of the caller's history/preferences would make sense. The Spanish option and the FedEx web announcements are irritants when you hear them once. This gets worse when you hear them every time that you call. Should not have to provide my account number, particularly not for an application like requesting a package pickup. The use of dated TTS technology really sticks out. The recorded voice is good and then you hear the address with this antiquated TTS. FedEx makes it fairly easy to get to an agent most of the time. At a number of points in the menu, they just block it out, though. I provided information to the system to calculate the cost to ship a package. When I went to the agent, she did not have this information. They do not provide a wait time estimate or offer a callback option when you go to the wait queue. From my experience, this is not necessary since FedEx invariably has adequate agents and wait queues are non-existent.

 

Dr. Walter Rolandi examination of FedEx's customer service line

The Examination Sessions

This month the patient is a FedEx speech enabled IVR. It can be reached at 1-800-GO-FEDEX (1-800-463-3339). The system offers seven (7) options on its Main Menu. My task was to call the system and find a nearby drop-off location. Only one call was necessary to perform the assigned task. However, a second call was made in order to test the system's error handling strategy.

Call One:

The call is answered with a brief greeting. Then comes a Spanish language option followed by approximately 1.5 seconds of silence.

Welcome to FedEx. Para continuar in Español, diga Español.

(1.5 seconds silence)

If you know the option you'd like, please say it now.

(1.5 seconds silence)

Please say: Schedule a pickup

Track a package

Find location

Get rates

Order supplies

Office and print services

Or International 

(1.5 seconds silence)

To hear these options again, say repeat.

To hear more options, say more.

The call required approximately 30 seconds to arrive at this point.

I responded, "Find location" to which the system said, "All right. Locations". The system then explained that it would use a zip code to find nearby locations and then prompted me to provide one. I complied, was thanked by the system, told that it needed some time to find the appropriate locations and then read a "by the way" prompt noting that locations can also be found at FedEx.Com.

The system played an "I'm working" earcon but quickly returned with three relevant locations. The location information used high quality (human) audio prompts that described the class (drop box, official station, etc.) and time schedules of each pickup location. TTS was used for the name and address of the locations.

Call Two:

The second call was the same as above except that I either said nothing or made intentional verbal errors after the Main Menu to see how the system would respond.

The Diagnosis

  1. Main Menu is too long.
  2. There are some minor timing issues.
  3. There are some pointless almost annoying "hints" as to what the user should say.
  4. Spanish option prolongs interaction by about 5.0 seconds.
  5. During Spanish option, dial through and barge in are not fully supported.
  6. DTMF is supported but the system does not inform the user of the feature. (At least it did not do so during the diagnostic calls).
  7. Minor anthropomorphic excesses are present.
  8. System contains a web ad.
  9. Some prompts were needlessly "conversational".
  10. Some prompts could be trimmed.
  11. TTS leaves a lot to be desired.

With the exception of Main Menu's length, this is a reasonably healthy system. Its health however could be significantly improved with some relatively minor changes. Pace was generally good but there are issues with timing, mostly after prompting for user input. The system prompts for a response but then begins speaking again (with pointless almost annoying "hints", no less) before the user has had time to do so. The Spanish option needs work and the web ad should be removed. The anthropomorphic and conversational excesses along with unnecessary prompt content would be easy to improve. TTS was at times difficult to understand and made at least one major mistake in reading the name of a particular location.

The Prescription

  1. Consider eliminating or subordinating some Main Menu options.
  2. Permit dial through and barge-in during the Spanish option or, preferably, totally remove the Spanish option. All callers would be better served if Spanish speakers were provided a dedicated 800 line.
  3. Tune timers.
  4. Provide some indication that DTMF is supported.
  5. Eliminate web ad.
  6. Eliminate existing "hints", anthropomorphic and conversational excesses and otherwise trim prompts of all unnecessary content.
  7. Consider using a better TTS engine.

The Prognosis

Very good to excellent.

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com

 


October 13, 2006

Walt & Walter's VUI View: on Comcast's customer support Telephone Self-service Line

The Caller's View from Walt

I called the 800-COMCAST number.  After asking me for the phone number that had the service (it did the same thing whether or not I was calling from the line or not) it then asked me to press 1 for English. It then gave me a menu that asked me if I was calling about a billing issue, a trouble, a sales inquiry or a move/change.  This does not make a bit of sense to me.  The caller that has a question about his bill is getting the same treatment as the caller that is interested in switching his telephone service and broadband to Comcast.  The caller that wants to purchase something gets another menu that asks him to further define what product he is interested in purchasing.  If this isn't enough aggravation, when you get to an agent, they don't have any of this information and you start all over again. I called the local Comcast number that was on my bill.  It seemed virtually identical to the 800-COMCAST line with the exception that it after the greeting, it started playing music and told me about the Comcast web site. It did pull my CallerID and asked me to validate it from the last four digits. This seemed reasonable until you realized that it had just wasted my time asking me to confirm my language preference and this was done after it told me about the web site in English.

 

Dr. Walter Rolandi examination of Comcast's customer service line

The Examination Sessions

The patient this month is that of a Comcast DTMF IVR. It can be reached at 1-800-COMCAST (1-800-266-2278). The task was simply to call the system and navigate to its Main Menu. A valid account number and zip code were employed to access the system. Two calls were required. Both were digitally recorded.

 

First Call

On the first call, I provided the account number when prompted. However the system responded with, "I did not recognize your selection". It then re-prompted me for the information which I again provided. Apparently accepting the account number, the system then asked for the zip code where the account is located. I entered the zip code but was again informed, "I did not recognize your selection". Upon re-prompt, I entered the information again. After about a 3.5 second pause, I was asked a question apparently intended to single out prospective sales calls and I was transferred to a sales representative.

Curious to understand what had happened, I played the digital recording of the call back through a DTMF decoder. The recording indicated that there was a sound quality issue during the first attempt to enter the account number. This would account for the initial failure. However the recording showed that the second entering of the account number and both zip code entries were correctly decoded and therefore presumably acceptable to the system. The system however, appeared to treat the call as an unidentified caller. The default response to such calls seems to be to treat them as potential customers and transfer them to a sales representative. The call required almost two minutes seconds to get to get to the point of transfer.

Second Call

On the second call, I provided the account number when prompted which appeared to be immediately accepted by the system. A transcript follows:

Thank you for calling Comcast.

 Please enter the telephone number including area code where you have or want service.

(I entered account number)

For English press 1. Para Espanol, oprima el numero dos.

(I entered DTMF 1)

Thank you. Please hold while we process your call.

(after 8 seconds of silence)

To help us better serve you please verify that the last four digits of your account phone number are 5 5 5 5. If this is correct, press 1. If this is not correct, press 2.

(I entered DTMF 1)

Please hold while we validate your information.

(after 4 seconds of silence)

For trouble with your service, press 1

Billing inquiries, press 2

Sales or sales information, press 3

All other questions including moves, changes to your service or discontinued service, press 4.

 

The call required 72 seconds to arrive at this point.

The Diagnosis

  1. I have no explanation for the failed first call. The telephone connection could have been so poor as to garble my DTMF entries. However the fact that 3 or the 4 DTMF entries were clearly decoded from an 8000 Hz, 16-bit mono recording of the call suggests otherwise. Thus the system may have difficulty decoding DTMF.

  2. The system asks the user for his language preference after he enters his account number.
  3. The system has what appear to be long and awkward silences.
  4. System feedback prompts are not particularly informative or helpful.
  5. Several prompts would benefit from trimming.

In summary, the first call raises the specter that this system may have serious problems. On the other hand, the second call suggests, at least when one's account number is recognized, that the system is not really all that seriously sick.

The Prescription

Cosmetic changes could address the problems in feedback and unnecessarily verbose prompts.

Reversing the order of the language preference question would also appear to be fairly trivial. Ideally, the question would never be asked more than once: the caller's language preference should be retrievable once he has provided his account number.

Eliminating or reducing the long silences may be more difficult if they are due to backend issues. Inserting an, "I'm working" earcon would help alleviate that problem however.

The Prognosis

Quite good, given a little TLC.

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com


September 13, 2006

Walt & Walter’s VUI View: on United Airline’s Reservation/Sales Telephone Self-service Line

The Caller’s View from Walt

Called the United Airlines 800-864-8331 reservation sales line. Up-front greeting that goes on and on. Has to tell me about the wonders of the United web site every time that I call. Announces that the call is being recorded as part of the main greeting even though I may never connect to an agent. Touch-Tone menu up front. Choices are: 1) flight status 5) Redeem frequent flyer miles; 3) U.S. travel reservations; and 4) International travel reservations. The frequent flyer mileage redemption option is a Touch-Tone one while the others are speech-enabled ones. The redemption option is the 2nd item on the menu, yet the instructions are to “press 5”. Pressing 2 also gets you to the redemption option which makes this strange. No CTI, so I have to repeat everything again to the agent and queue wait times are not provided. The response to requesting an agent is to connect to self-service speech-enabled self-service reservation line. If you continue to insist on being connected to an agent, the self-service system finally stops arguing with you but not until you tell it if you are looking to travel within the United States or international. When I get to an agent, I was connected to ones in India with thick accents that were difficult to understand. This is primarily a sales line. If you have a lost-bag problem, United has a separate line for this. What possesses these folks to think that it makes any sense to make it at all difficult for a paying customer to get to an agent and buy something? United also lies to the caller. The final phrase of the main menu is: If you are calling from a rotary dial phone, please stay on the line and your call will be answered by a United Airlines sales representative. What really happens is that if you don’t respond at all, you get connected to the speech-enabled self-service system which then requires that you to argue with the self-service before you get connected to a human.


 

Dr. Walter Rolandi examination of United Airlines customer service line


 

The Examination Sessions

Walt Tetschner suggested that I obtain the status of a flight using the United Airlines IVR. My task was simply to call the system, navigate to the point where I could obtain flight status information, get the information and hang up.


 

Completing this seemingly simple task proved to be quite an ordeal entailing three separate calls and requiring almost 13 minutes.

 

Call Commonalities:

All calls are apparently answered by a routing application. It first greets the caller, entreats him to visit the United web site (a.k.a., reads a “web ad”) and then warns the caller using a stereotypical statement concerning calls being recorded “for quality assurance purposes”.


 

The caller then hears the Main Menu:


 

For flight arrival and departure information please press 1

To redeem miles from your Mileage Plus Account, please press 5

For travel within the 50 United States, please press 3

For international travel including Canada and the Caribbean, please press 4

If you are calling from a rotary dial phone, please stay on the line and your call will be answered by a United Airlines sales representative.


 

The system requires 57 seconds to arrive at this juncture. At times, the voice talent sounds eerily like Kermit the Frog.


 

First Call

On my first call, I entered DTMF1 after the initial 57 second opening. I then heard, “Welcome to United Airlines flight information line” followed by almost 17 seconds of information about the system and instructions as to how one should interact with it. I did not have a particular flight number in mind so I followed the instructions (saying, “I don’t know it” when prompted) and then completed the laborious process of providing and confirming the departure city, arrival city, time of day, etc. for a flight from Columbia, SC to Washington, DC. The system collected all of this information and completely restated it, asking me to confirm that the information was correct. When I did so, it returned with departure information on a Flight 7197.


 

Out of curiosity, I looked up Flight 7197 on United’s website. To my dismay, I found that the flight did not fly from Columbia to Washington but rather from Columbia to Chicago.


 

Second Call

I decided to give the system another chance using a known flight number (Flight 7279) which I obtained from United’s website. I called and again listened to the 57 second opening. Since I had just interacted with the system using speech, I inadvertently said, “One” in response to the Main Menu (above). The system apparently treated the utterance as a “DTMF non-response” and after approximately six seconds of silence, my call was transferred.


 

Note that the Main Menu states:


 

If you are calling from a rotary dial phone, please stay on the line and your call will be answered by a United Airlines sales representative.


 

This is not what happened to me however. My call was transferred to the United Airlines reservation system. I attempted repeatedly to escape back to the Flight Status system but to no avail. Eventually, I hung up.


 

Third Call

I called the system again, entering DTMF1 after the 57 second opening. When prompted for the flight number, I said, “seventy two seventy nine”. The system said,


 

I think you said, “seventy two seventy nine”. Is that correct? Please say yes or no.”


 

I answered, “yes” and then, when asked, requested departure information. What then followed was incomprehensible to me. The system seemed to give conflicting and thoroughly confusing departure and arrival information. Upon hanging up, I went back to the website where I discovered that “Flight 7279” entails to two separate flights. It appears to be a shuttle that departs from Washington, arrives in Columbia and then returns to Washington. I suppose this is why the system reports confusing arrival and departure times.


 

The Diagnosis

  1. The system is fundamentally flawed: it provided information on an incorrect flight (7197), it failed to transfer me to a “sales representative” and, presumably due to the flight naming conventions of United Airlines, it provided incomprehensible flight departure and arrival information (7279).

  2. The system contains a web-ad.

  3. From the user’s point of view, the system mixes modalities, initially requiring DTMF while supporting speech and DTMF after the Main Menu.

  4. The system seems to talk interminably.

  5. The system annoyingly and repeatedly tells the user exactly what to say.

  6. There is far too much unnecessary prompt content.

  7. The system pointlessly attempts to sound conversational.

  8. Many persona excesses and anthropomorphisms are evident.

  9. The system ineffectively attempts to provide alternative but complicated and painstakingly slow methods to find flight status information.

  10. The time required to complete a simple flight status query task is unnecessarily long.

  11. The flow, production quality and professionalism of the system’s prompts are at times questionable.


 

In summary, this system is gravely if not fatally ill.


 


 

The Prescription

While some cosmetic changes could improve, or at least shorten, the user experience, this system needs major surgery. Again, after the system confirmed that my query regarded a flight from Columbia to Washington, it then provided information about a flight from Columbia to Chicago (see Addenda). This implies basic logic or data integrity issues. The system says that calls will be transferred to sales representatives when they are actually transferred into another automated system from which there is no apparent escape. It is difficult to understand why more than one flight can have the same flight number (see Addenda). Both of these issues imply faulty business decisions.


 

Cosmetically, the following treatments could help:

  1. Universally support speech.

  2. Eliminate all unnecessary prompts and prompt content. Aggressively trim all prompts.

  3. Eliminate all preemptive prompt instructions.

  4. Eliminate persona and anthropomorphic excesses.

  5. Simplify and speed up the overall process.

  6. Fix production quality problems


 

The Prognosis

Unless the systemic problems can be surgically excised, the prognosis is far from good.

Addenda

Airline

Flight

Departure

Arrival


 

Frame1

Flight 7197

Columbia, SC (CAE)

Chicago, IL (ORD)

United

Operated by UNITED EXPRESS/MESA AIRLINES

Mon, Sep 11

Mon, Sep 11

 

NOT DEPARTED

Scheduled: 

5:34 PM

Scheduled: 

6:45 PM

 

Details

Estimated:  

5:34 PM

Estimated:

6:45 PM

 

 

Reason: --

Reason: --

 

 

Gate: --

terminal 2 concourse f

 

 

 

Gate:  F1

 

 

 

Baggage claim:  3

Airline

Flight

Departure

Arrival


 

Frame2

Flight 7279

Washington, DC (IAD)

Columbia, SC (CAE)

United

Operated by UNITED EXPRESS/MESA AIRLINES

Fri, Sep 8

Fri, Sep 8


 

 

NOT DEPARTED

Scheduled: 

5:00 PM

Scheduled: 

6:22 PM

 

Details

Estimated:  

5:00 PM

Estimated:

6:22 PM

 

 

Reason: --

Reason: --

 

 

concourse a

Gate:  --

 

 

Gate: A4

Baggage claim:  --

 

Flight 7279

Columbia, SC (CAE)

Washington, DC (IAD)

 

Operated by UNITED EXPRESS/MESA AIRLINES

Fri, Sep 8

Fri, Sep 8

 

NOT DEPARTED

Scheduled: 

7:00 PM

Scheduled: 

8:26 PM

 

Details

Estimated:  

7:00 PM

Estimated:

8:26 PM

 

 

Reason: --

Reason: --

 

 

Gate: --

concourse a

 

 

 

Gate:  A2

 

 

 

Baggage claim:  1


 


August 12, 2006

Walt & Walter's VUI View: on Verizon's Customer Self-service Telephone Self-service Line

The Caller's View from Walt

I've been a Verizon customer for as long as I can recall. I remember when they were known as New England Telephone. This was before the breakup of the original AT&T (1984), and then it became part of NYNEX which finally merged with Bell Atlantic, acquired GTE and became Verizon. I had a noise problem on one of my phone lines so I decided to call Verizon to get it repaired. To determine the number to call I looked on my monthly phone bill. Aha! I quickly found that it listed a separate number for Repair. I called the number and the first thing that it dumped on me was a menu that asked me what I was calling about. It gave me an option via the menu to inquire about most anything (billing, service changes, product information, etc. in addition to repair) What is wrong with these people? They specifically identify a number that you should call for repair. When you follow their instructions, they turn around and ask you what you are calling about. This is one of the most irritating things for a caller. The caller took the time to figure out what the proper number to call is. For this, they get rewarded by having their time wasted having to again explain what they are calling about. The difference between the Verizon Portal (which Walter called) and the published Verizon Repair number is also interesting. The Verizon Repair number greets you by telling you how to get to Verizon's web site. This is so important to them that you can't interrupt it and are forced to listen to it. The Verizon Portal greets you with an advertisement claiming that they are my broadband company, which, unfortunately, I'm already aware of. All that is needed and appropriate is a simple "Verizon". The Verizon Portal captures your number via CallerID while the repair line makes you enter your 10-digit phone number. For the Repair line menu, if you select an option other than Repair, you are prompted with: " Are you calling about 508-551-0413? This is not even the correct area code for my number. I called it and found that it was another Verizon Repair line that functioned differently than the Repair line that was listed on my bill. No CTI, so you need to repeat everthing that you entered when you get to a human. Also without CTI they can't accurately tell you how long the wait in the queue is. Just have to listen to the Verizon ads without knowing how long the wait will be.

Dr. Walter Rolandi examination of Verizon Self-service

The Examination Sessions

I called the Verizon IVR a week or so after Walt Tetschner had used it to report a problem with his phone service. My task was simply to navigate to the point where I would report a problem. I called Verizon toll-free number (800-870-9999) twice and recorded both interactive sessions. Normally only one call would have been necessary however something completely unpredictable occurred on the first call that obliged be to try again.

First Call

The system answers with a brief greeting (approximately 4.7 seconds):

Good morning. Thanks for calling Verizon, your broadband company.

The greeting is followed by a three-note audio-icon. After about 4.0 seconds of dead air, the system apparently transferred the call to some other system. After five rings, a system answered saying,

You have a choice of regional toll and long distance providers....

(approximately 26 seconds of some sort of sales pitch)

Then came another ring after which a human answered. I explained to the nice lady that I didn't know how or why I was transferred to her, apologized and hung up.

Second Call

Again, the system answered with the same brief greeting:

Good morning. Thanks for calling Verizon, your broadband company.

It then read out the number that I had called from asking if this was the number I had called about. I answered, "No" and after an awkward (about 3.0 seconds) pause it said,

Ok. What's the phone number you are calling about?

I provided a valid Verizon number that the system then read back to me (after an almost 4.0 second pause) for confirmation with a yes/no question. I responded, "yes" in less than 1.0 second but as I was speaking the system played a prompt instructing me to say "yes".

The system then played an "I'm working" audio con for a little over 6.0 seconds after which it announced,

I've got your records.

shortly followed by the open-ended prompt,

In just a few words, how can I help you today?

Before I could respond, the system said,

For example, you can say something like I have a question about a long distance charge on my bill.

Given my task to find information about repairs, I finally said,

I'm calling about repairs.

The system then asked if my trouble was with my,

...DSL service, voicemail, phone line or none of those.

When I answered, "None of those", I was transferred to:

"...the automated system that can help you with that."

It is unclear what then happened. What followed was a series of unnecessary instructions, another read-back of the number I had called about and another announcement that my account history had been found.

Eventually, another open-ended prompt invited me to briefly describe my problem or ask for a list of possibilities.

I said, "List" and then hung-up after "six common types of problems" were read out.

This entire process required 202 seconds. I experienced either two or three confirmations depending upon one's interaction of the semantics of the last telephone number read-back.

The Diagnosis

  1. The first call was bizarre.
  2. Regarding the second call, there is unnecessary prompt content.
  3. The system ineffectively attempts to sound conversational.
  4. Some persona excesses and anthropomorphisms are evident.
  5. The system has serious timing problems.
  6. The system has serious turn taking problems.
  7. The system occasionally appears to be unnaturally slow responding (presumably a backend issue).
  8. Open-ended prompts provide insufficient feedback.
  9. Many prompts contain confusing preemptive examples of what the user is supposed to say.
  10. The time required to complete the task is painful and excessive.
  11. The production quality of the system's prompts is not as good as it might be.
  12. The voice talent on some prompts occasionally sounds over-animated while at other times sounds almost sedated.
  13. Although it is not clearly indicated, DTMF is supported throughout the system except at the open-ended prompts.

I may never know what happened on the first call. Perhaps it was some bizarre fluke or perhaps the system reacted inappropriately to something I did not even say. Whether fluke or otherwise, this experience could signify a substantial flaw in overall system architecture. Who knows?

Once accessed, the system is not as bad as it could be (that is, assuming that my transfer to "...the automated system that can help you with that." was logically correct and not actually due to a false positive response from the recognizer). In any event, the system has serious usability issues. The timing problems and preemptive "help" prompts make it difficult to discern when to speak. The system can also appear "rude" when it speaks while the user is speaking. It also has a good bit of unnecessary or pointless prompt content that only serves to inform the user that the system is wasting his or her time.

The Prescription

While seriously ill, this system could be put into excellent health using the following treatments:

  1. Trim all prompts.
  2. Tune timing and turn-taking variables.
  3. Eliminate unnatural (presumably) backend delays.
  4. Eliminate persona and anthropomorphic excesses.
  5. Make the overall process faster.
  6. Consider eliminating open-ended prompts. Otherwise, provide more meaningful feedback.
  7. Fix production quality problems

The Prognosis

Pretty good, assuming the turn-taking and timing problems are not insurmountably linked to back-end or infrastructure issues.


July 7, 2006

Walt & Walter's VUI View: on CareMark's Customer Self-service Telephone Self-service Line

The Caller's View from Walt

I've been refilling prescription drugs via mail order for a number of years. First with AdvanceRx and now CareMark who acquired AdvanceRx. Price is much lower than buying from the local drugstore. I've been calling them regularly and doing the same thing each time. Ordering the same prescription drugs, paying for them with the same credit card and having them sent to the same address. They have all the ugly stuff that callers hate. Spanish options to waste their time. No CTI, so they can't tell me how long the wait queue is and I have to repeat everything to the agent. Prescription refill is a fine application for self-service. The caller knows precisely what they want to do. They do it over-&-over – usually from the same phone. Despite this, the CareMark prescription refill self-service application is a pretty ugly implementation and probably has a relatively low usage. Lots of really dopey stuff like telling the caller that they can say Goodbye. When I say my account number, I need to say "zero", since it does not recognize me saying "oh".


 

Dr. Walter Rolandi examination of CareMark Self-service

The Examination Sessions

I called the CareMark IVR the day after Walt Tetschner had it to order some prescriptions. My task was to "check on the delivery status" of the prescriptions. Accordingly, I called CareMark toll-free number (888-424-6618) and recorded my interactive session. Only one call was needed, given the task at hand.

The system answers with a brief greeting (approximately 3.3 seconds):

Thank you for calling CareMark Prescription Service.

A simple Main Menu then follows:

To refill a prescription, press one

To check the status of a mail order prescription, press two

For all other enquiries, press three

Para escuchar este mensaje in Espnaol, por favor oprima quattro ahora.

(To hear this message in Spanish please press four now)

The system requires only 16 seconds to hear its Main Menu.

Given the task was to "check on the delivery status" of the prescriptions, I entered DTMF2. After approximately four seconds of "dead air", I heard the sound of a ringing telephone which was immediately followed by the prompt:

How would you like me to find your order?

By prescription number

Most recent order

Or by drug name

Otherwise say, repeat, goodbye, representative or press zero.

I followed the instructions and completed the task in four minutes and forty-nine seconds. I experienced two confirmations and three speech recognition failures, two of which were at yes/no junctures.

The Diagnosis

  1. The system mixes interactive modalities. Its Main Menu system is DTMF only while the "delivery status" system is speech only (although it does permit pressing zero for a representative).

  2. Most prompts are verbose and could be trimmed.

  3. The system is at times pointlessly anthropomorphic.

  4. The system has severe turn taking and timing problems.

  5. Many prompts contain confusing preemptive examples presumably intended to help the user know what to say.

  6. The system supports various methods of tracking prescriptions that might be intended to help certain classes of users but may in fact only confuse and complicate the process for them and others.

  7. The "delivery status" task process is unnecessarily complicated.

  8. The time required to complete the task seems excessive.

  9. The Spanish option is probably unnecessary and, in any event, ineffectively deployed.

  10. The system supports saying "O" for zero in some places but not in others.

  11. The production quality of the system's prompts is inconsistent: prompt recording quality and volume vary in some states.

While this system does exhibit an appreciation of many VUI design best practices, it nonetheless suffers from some severe infirmities. First is that the system takes a task that should be very simple to perform quickly and complicates it unnecessarily. Second, mixing modalities and supporting the use of "O" for zero in some places but not in others can thoroughly confuse users. Third, the systems turn taking and timing problems make the dialog seem needlessly odd. They additionally prompt user input inappropriately and ineffectively.

The Prescription

While seriously afflicted, this system could be put into excellent health relatively easily. The following treatments would be required:

  1. Simplify the process for obtaining "delivery status"

  2. Eliminate modality switching

  3. Eliminate the need to restrict the use of "O" and zero

  4. Fix all turn taking and timing problems

  5. Eliminate unnecessary prompt content

  6. Support Spanish callers more effectively

  7. Fix production quality problems

The Prognosis

Quite good, given the will to improve.


 


June 3, 2006

Walt & Walter's VUI View: on Amtrak's Customer Self-service Telephone Self-service Line

The Caller's View from Walt

The Amtrak customer support number is 800-USA-RAIL. The first thing that you are greeted with is Julie Amtrak telling you that about the Amtrak web site and how the fares may be lower. To torture you more, the caller is forced to listen to this in its entirety, since overstrike is enabled. This gets even more irritating each time that you are forced to listen to it. I have used the Amtrak web site a number of times and never found a lower fare. The attempt to create an anthropomorphic agent is a failure. Did you ever hear a real person say: "did you mean?" when they did not understand what was said? The little phrases (hold on, okay, got it, great) that are supposedly intended to help establish Julie as a real person, become irritants since they add nothing and waste the caller's time. Even after using the Amtrak self-service a number of times, I found that it took too long to accomplish a task. The Amtrak live-agent was invariably able to handle it much faster. Since the Amtrak hold queue for an agent is usually non-existent, it makes most sense to simply go to a live agent up front and not deal with the self-service. Amtrak needs a Work Force Management (WFM) system. They are over-staffing their agent positions and have eliminated the need for a self-service capability. This situation seems perfectly consistent with the management of Amtrak. Wasting money on unnecessary things and not investing in essential thngs such as adequate train maintenance.

Dr. Walter Rolandi examination of Amtrak's Julie

The Examination Sessions

I called the Amtrak toll-free number (800-USARAIL) and recorded three sessions. As with all VUI Reviews, my first objective was to simply navigate through the system's Main Menu. My second call was intended to test the system using a speakerphone. The objective for the third call was to comply with Walt Tetschner's request that I "check the status of a train".

First Session:

The system ("Julie") answers with a brief (approximately 3.0 second) greeting:

Hi. I'm Julie, Amtrak's automated agent.

The greeting is followed by an approximately 6.0 second web ad:

I invite you to visit our website, Amtrak.Com. Lower fares may be available.

A wordy Main Menu then follows. Simplified, its choices are:

Train Status

Schedules

Reservations

Agent

The system requires only 24 seconds to hear its Main Menu.

Second Session:

Recall that this call was made to the system on a speakerphone. I said nothing to the system and made no noise at all during the call. The system behaved precisely as before until it was well into the Main Menu whereupon it began to experience repeated recognition failures. It was sufficiently intelligent however to suggest that the errors may be due to a "weak cell phone signal", "speakerphone echo or some other background noise". The system quickly transferred the call to an agent.

Third Session:

The last session was essentially equal to the first up to the point where I was prompted to say, Train Status. I attempted to get the status on a train leaving from Columbia, SC to Washington, DC. After a lengthy interrogative process, the system told me that it had found two trains. The first was described as a direct train called the "Silver Star" and the second, also (oddly) called the "Silver Star", was said to involve a connection through Richmond. I was unable to resolve the ambiguity and obtain the status. The system eventually transferred me to an agent.

The Diagnosis

  1. The system is presumptuously anthropomorphic
  2. The system is unnecessarily animated
  3. Most prompts contain unnecessary prompt content
  4. The Web ad annoyingly wastes of the user's time
  5. The time required to hear the Main Menu could be reduce by 40%-50%
  6. The "Help" feature is unhelpful and unnecessary

The Prescription

My failure to complete the task was presumably due to a functional problem of the system and thus did not figure largely into my assessment of the VUI. In fact, all things being considered, this system exhibits a solid appreciation of VUI design best practices. For example:

  • Its Main Menu includes only four choices, one of which is to speak with an agent.
  • It is easy to reach an agent either verbally or by entering DTMF 0 at any point after the web ad.
  • The system supports a simple DTMF backup scheme behind its menus.
  • Its error logic presumably checks for evidence of "weak cell phone signal", "speakerphone echo or some other background noise".
  • A Spanish option is offered only after a transfer has been initiated.
  • And overall production quality is high.

In summary, the VUI is in pretty good health but it does however suffer from some behavioral problems and a mild personality disorder. It could be easily improved by shortening the greeting, eliminating the web ad, eliminating the help feature, trimming most of its prompt and finally, de-animating its anthropomorphic persona.

The Prognosis

The cure is simple. Improvement is all but assured.

 


May 3, 2006

Walt & Walter's VUI View: On the United States Postal Service (USPS) Customer Support Line

The Caller's View from Walt

The USPS support number is 800-275-8777. My local post office has stopped listing their local number in the phone book. As a consequence, I have to call a nationwide 800 number to obtain information about the local post office. After being told how to get a Spanish menu and instructed how to get tax forms from the IRS, a 4 item menu was presented. I could get a zip code, mailing rates, track & confirm a delivery, or do something with my delivery services. Since I wanted the hours of the local post office, and none of these gave me this, I selected the "More options" choice. I now found that I could change my address, schedule a pickup, buy stamps, and finally get Post Office Information. When prompted, I provided my zip code and the system spoke back the location of my post office, the hours and the telephone number. Despite the shortcomings, the postal service system worked reasonably well. The Spanish option and the IRS forms announcement were a irritating but they did it so quickly that the pain was not that huge. I guess that when you are a legal monopoly, such as a government post office, you get pretty efficient at abusing people. The order of the options did not make a lot of sense. I can't imagine many folks calling to: obtain a zip code, get a mailing rate, or track & confirm a delivery. Yet - stopping and re-starting delivery is something that is commonly done. The system does not have CTI implemented, so that it could not provide a valid queue wait estimate and I had to provide all of the information all over again to the human that I finally talked to. When I tried to get to an agent, the system at first ignored me. Then it insisted that I provide information that identified what I was calling about This became even more irritating when I got to the agent and found that the information had not been provided to the agent. The system putting so many options on the menu makes everything confusing and difficult to use. Attempting to automate this many functions is a misuse of the technology. They should get rid of all but the top 2 or 3, in terms of usage.

Dr. Walter Rolandi examination of USPS VUI

The Examination Session

I called the USPS system at 1-800-275-8777 two times. My first objective was to simply navigate to the systemҳ Main Menu. The objective of the second call, as per Walt Tetschnerҳ request, was to get the location and hours of operation of my local post office. Although the systemҳ Main Menu is tiered, getting to and hearing the first or main component of the Main menu took only 29 seconds.

The system answers with a brief greeting followed by instructions for Spanish speakers. The Spanish language option adds 4 seconds to the call. The system then played a 10 second message about tax forms and how to contact the IRS. Thus while it only takes 29 seconds to hear the first tier of the systemҳ Main Menu, 17 of those seconds are consumed by the greeting, the Spanish instructions and an IRS message.

The first tier of the Main Menu is as follows:

Find a zip code

Get mailing rates

Track and confirm

Delivery services

Users are also encouraged to say, "More options" whereupon the following choices are provided:

Change of address

Schedule a pickup

Buy stamps

Say Post Office Information (for post offices hours and locations)

Finally, the system encourages users to say, "Other information". When I obliged, the system asked me if I was calling about "passport information". I answered, "no" and was transferred. The events in this session required 67 seconds to complete.

The second session was essentially equal to the first up to the point where I was prompted to say, "Post Office Information". Saying "Post Office Information" provided no echoic feedback. It did however produce two beeps that suggested being transferred followed by 5 seconds of dead air. The system then asked me to specify, "My Post Office" or any other post office. It allowed only 2 seconds to respond however and interrupted me as I said, "My Post Office". This generated a recognition failure, replayed the prompt and obliged me to repeat my request.

This branch of the system seemed to employ either mixed female voices or have quality issues with some of the prompts. The actual information about "My Post Office" was read out rather abruptly in a male voice.

The system required 133 seconds to complete the task.

The Diagnosis

This is a relatively healthy VUI, particularly when one considers just how sick most others are. It does suffer from one significant problem (#1 below) but most of its other issues relate to its production quality.

  1. Like many IVR systems, the system offers too many choices.
  2. At least one turn-taking timing variable needs to be adjusted.
  3. Some prompt content, (particularly within the initial 17 seconds) is unnecessary.
  4. The system apparently uses an odd and uninformative audio-icon for user feedback.
  5. There were pointless silences in several areas of the dialog.
  6. The system mixes voice talent in an apparently meaningless way.

The Prescription

This system is actually in pretty good shape. It could be even better however if it were to shed a few of the least requested tasks, trim a few prompts, tune some timer variables, provide more meaningful feedback and use voice talents in a more consistent fashion.

The Prognosis

Improvement is possible and quite likely.

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com

 


April 6, 2006

Walt & Walter's VUI View: on AOL's Customer Support Line

The Caller's View from Walt

The AOL support number is 800-827-6364. This is also the line for new subscriber signup, for upgrading from dial-up to broadband and for service cancellation. Does not answer until the 4th ring. After the greeting (1st female voice), you get an emergency announcement (male voice that sounds like the same voice that BofA is using) that a release of Norton anti-virus has a bug and that you should call another number if you are using Norton anti-virus software. I don't use Norton anti-virus software so this was just a waste of my time. Using the same number for functions that are totally separate is a fundamental mistake. The first thing that they do is to waste the caller's time by getting them to identify which of the three things they are calling about and then re-routing them. Using DNIS would be an efficient way of doing this. Aside from not irritating the caller by wasting their time, it would cost AOL less since the telephone connect times are reduced. Don't use caller ID to obtain my telephone number which they never use anyway. After you identify that you are a subscriber by saying "yes" to the question: "are you a subscriber", the system then gathers additional information such as your phone number and the first three letters of your screen name. Despite never actually using any of this information for anything, it wastes your time by laboriously reading it back to you a letter at a time. Finally, you get to an eight item menu. Despite having wasted time identifying that I was a subscriber and providing my phone number and screen name, the menu includes items for a non-subscriber to sign up. I found that you can avoid having to provide the unused telephone number and screen name by telling the system that you are a non-subscriber. Actually, for technical support calls, the fastest way is to just keep hitting the 0 key. Amazingly, an item that they include on the menu is service cancellation. After you select technical support from the menu, you need to identify the sort of computer that you are using and the broadband connection that you have. Finally, you have to state what the problem is. I've found that whatever you state as the problem, is interpreted as an AOL connection problem. Don't argue with it &ndash just agree that it's an AOL connection problem. Doesn't matter since none of this information is used for anything. These folks are just jerking the caller around. Finally you get put into a queue. AOL doesn't have C-TI implemented which means that they can't estimate the wait time with any precision so they just have the guy with the BofA voice announce that it will be at least so many minutes. You now get 10-20 minutes of music interrupted every few minutes by the BofA guy informing me of another wonderful feature that AOL has to offer. Finally, I get to the agent. She does not have any of the information that I had provided to the IVR. Unfairly - I dump on her. No wonder that the agents hate the IVR. Shortly after every contact with AOL customer service, you receive an e-mail survey from AOL which claims to attempt to find out how you liked your customer service experience. This survey is a joke. The only thing that it asks questions about is the performance of the agent. AOL pretends to be unaware of how really awful their telephone self-service really is. They pretend to care about their subscribers by beating on the weakest and least able to defend themselves.



 

Dr. Walter Rolandi examination of AOL VUI

The Examination Session

I called the AOL system using account login information provided by Walt Tetschner and navigated to the system's Main Menu. This took 181 seconds. After an initial greeting, the system plays a 60-plus second prompt about a problem apparently caused by Norton Utilities. It basically instructs callers who think they are having this problem to call another number. Spanish is then offered to the caller (after over a minute of unsolicited English).

Before arriving at the Main Menu, the system asks the caller if he is an AOL member, inviting the caller to say, "yes, no or I don't know". I failed to respond when prompted twice, causing two no-response or timeout messages to be read. I then provided a phone number and some alphabetic information in order to complete the login process. In all, I made five seemingly appropriate utterances, experienced two recognition failures and one confirmation.

After listing its six Main Menu choices, the system prompted that "Help" was available if further explanation were needed. Saying, "help", initiates a 45 second prompt which essentially restates the system's Main Menu choices. Note that the first choice is to "sign up" with AOL.

The session, up to this point, required over 225 seconds.

The Diagnosis

A substantial number of VUI infirmities and functional inadequacies were evident:

1. The overall system organization seemed irrational.

2. The system had difficulty detecting speech acts.

3. Error timers seem unnatural.

4. Some menu options appeared irrational.

5. Many, if not most prompts were unnecessarily long.

6. The system employs an odd audio-icon for user feedback. It does so inconsistently and ineffectively.

7. The system was unbearably animated.

8. The system is far more complicated than it need be.

9. The system provides insufficient movement for the user towards task completion and therefore leads to user frustration.

10. "Help" was of no help.

11. The system seems structured in such a way as to painfully discourage transfers to humans.

12. Even though the system tells the user that he will soon be transferred, it repeatedly interrogates the user for additional information.

The Prescription

This system is a mess. It may suffer from basic inabilities to detect user speech acts. It also appears to be stupid: after confirming that the user is an AOL user, the system offers the user the option to become an AOL user. This is basically irrational. Trimming prompts, eliminating anthropomorphic approximations and other cosmetic changes could improve the system somewhat but its organization is fundamentally flawed. Much more substantial changes would be necessary to overcome its profound shortcomings.

The Prognosis

Improvement is possible but it is not likely to result from a casual intervention. Major surgery is probably required.

Contact: Walter Rolandi, Ph.D., The Voice User Interface Company, LLC +1. 803.252.9995, wrolandi@wrolandi.com www.wrolandi.com



 

 


March 3, 2006
 

Walt & Walter's VUI View (WWVV): on Bank of America's Consumer Telephone Banking Telephone Self-service


 

The Caller's View from Walt

Back in the July, 2005 issue, we reported on the lousy speech-enabled implementation that BofA has foisted on me. At that time the system was relatively new. Over 6 months have gone by and the excuses that it is having birthing pains and the problems will be resolved soon, have become a bit lame. Everytime that I call, I still get provided with detail instructions on how to disable the speech service. I called them on February 22nd (two full days after the holiday) and they were still including a special message telling me that they were closed on the holiday and that my account information would be updated after the holiday. CallerID is not implemented, so they require me to identify myself by entering my Social Security Number, which they call my Access ID. After I've followed their directions and disabled the speech recognition (managed to do this without crashing the car), I find that it's really difficult to enter my 9-digit Social Security Number and continue driving safely. After I get out of self-service and go to a human (BofA calls them Specialists), they do not have any information re the information that I provided to the self-service. Of course, without C-TI, they don't know how many calls are in the queue, so they can't inform me how long the wait will be. What is wrong with these people? They implement speech but not C-TI? Just ask the callers what they want. Answer is always the same: They want a quick-&-easy experience! The lack of any intelligent telephony in this application is appalling. Using CallerID to recognize who I am would eliminate the need to provide my Social Security Number which saves me time and is safer while driving. Having to repeat information to the agent that I have previously provided to the IVR is really irritating and wastes more of my time. The entire BofA telephone self-service application is slow-&-hard. Long and unnecessary instructional prompts. I've been using this system since last July. It hasn't improved since then which would suggest that it has limited adaptive capabilities. It would also suggest that BofA has little real interest in meeting the needs of their callers (customers). This is even more ugly because the speech overstrike is disabled and you need to listen to all of these long prompts. Doesn't recognize "natural numbers" being spoken. During the wait queue, BofA has decided to dump an advertisement on the callers. Captive audience &ndash no place to go! I don't at all remember what the advertisement was all about which suggests how effective it is. Some things improve with age. This telephone self-service application from BofA is not one of them. The BofA phone number for consumers to access their account information is 800-841-4000. This can also be accessed by calling 800-900-9000 and selecting option 4.


 

Dr. Walter Rolandi examination of Bank of America VUI:

The Examination Session

I called the BoA system using account login information provided by Walt Tetschner and navigated to the system's Main Menu. This took 99 seconds. I made seven situationally appropriate utterances, experienced three recognition failures and one confirmation. The entire session lasted 125 seconds.

The Diagnosis

A number of VUI infirmities were evident:

  1. The overall pace of the system was painfully and artificially slow.

  2. Some intra-prompt timers were inappropriate and thus caused turn taking errors.

  3. Several menu options offered unnecessary, unnecessarily complicated or confusing choices.

  4. The system provides excessive instructions, presumably intended to help the user.

  5. Persona excesses were evident.

  6. The system's speech recognition recovery strategy is inefficient.

  7. The system ineffectively models the responses it seeks from its users.

  8. The system's speech confirmation strategy is questionable and can appear silly.

My isolated confirmation experience tells the "silly" story: After offering the Main Menu choices, the system eventually suggests:

System: "To end this call you can say, "goodbye" or just hang up."

Obligingly, I said, "Goodbye" to which the system replied:

System: "You said, "goodbye". Is that correct?"

Here is a prime example of how pretending to be "natural" or "conversational" can produce silly and annoying results.

The Prescription

This system could be considerably improved by adjusting timer values and better turn taking modeling. Over twenty-five of the 99 seconds required to hear the system's basic Main Menu choices consisted of "dead air" between individual system prompts and between system responses to user utterances. At least a third of this could be eliminated. This would move the user through the system faster and shorten the overall mean call duration.

Several menus could be significantly improved if simplified and/or shortened.

Almost all prompts are longer than they need to be. Most system prompts would benefit from a decisive application of Rolandi's Razor. (http://www.speechtechmag.com/issues/9_4/human_factor/).

All attempts to affect a human-like personality should be eliminated as well as most of the system's expressions of gratitude.

Finally, the system would greatly benefit from a less conversational and more informative error recovery strategy.

The Prognosis: Significant improvement is likely. Walt adds that this is only so if BofA has the good sense to realize that providing poor customer service as they are doing is not good business and invests in the repairs that are necessary.

Contact: Walter Rolandi, 803-252-9995, wrolandi@wrolandi.com

 

 

 

 

 

(c) 2008-2017 asrnews.com