www.earthlink.net myEarthLink myVoice My Account Support webmail.earthlink.net

« April 2008   |   Back to MAIN   |   June 2008 »

View The Web Mail Survey ResultsGeneral

From: Email Guy
Subject:       The Web Mail Survey Results
Date: May 8, 2008 3:04 PM
Permalink   |   Comment About This

Thanks to everyone who participated in our recent survey.

I received thousands of responses, and statistically the sample size indicates a 99% confidence level that the quantitative responses (multiple choice) fall within +/- 2.5% of the mean for the entire Web Mail user population. That means the answers should fairly represent most users. There was also great participation in the open-ended questions, where you could describe what you need or would like to see in Web Mail. I added all those answers up too (thousands of open-ended answers and I read every single one) and there were a lot of repeated themes that I'll summarize and rank.

As promised, I'm publishing the full results here. We're still digesting all the information internally and formulating plans to try and better meet the top needs of our users. But you can count on a lot of this information leading to specific solutions.

The survey gathered 3 sets of useful data:

  • Web Mail usage by tenure, frequency, connection type, browser, and OS.

  • Web Mail satisfaction for reliability, speed, features, storage, and ease-of-use.

  • Unmet needs and desired improvements.

We asked 8 required multiple-choice questions for quantitative and ranking data, and 3 optional open-ended questions for additional data on user needs and problems. Here's the results, and some comments summarizing what we learned:



First, the basic questions about Web Mail users.


q1.gif

q2.gif

q3.gif

If we count all the respondents who say they have used Web Mail for over 3 months, AND use Web Mail at least 3 times per week, AND use Web Mail at least as regularly as POP (Outlook or similar software), we get 77% of all the respondents. That's a high percentage of regular and heavy users providing the data. This "self-selection bias" of respondents means we are hearing from engaged users who have an opinion.


q4.gif

28% say they "usually" or "regularly" use Web Mail over dialup. This is a very important group that has specific needs, particularly speed optimization (more on that later).


q5.gif

q6.gif

  • 15% use Safari, which is a unique browser shipped on Macs that sometimes has different behavior than IE and Firefox.
  • 19% use Mac OSX.

  • 9% are using a Windows version older than XP or Vista (at least sometimes, users could choose more than one answer).

These are significant population segments that must be supported.



Next we asked users about general satisfaction.


q7.gif


I've graphed the responses. Here's the results for all respondents combined.

satisfaction-all.gif


Here's the results for only those users who said they usually use Web Mail over a high-speed connection in question #4.

satisfaction-hs.gif


And for the dialup users:

satisfaction-dial.gif

Both high-speed and dialup users rated "speed" and "features" the lowest overall score of the categories listed. The best overall scores were for "reliability" and "ease-of-use".

High-speed users rated features lowest.

Dialup users rated speed lowest.

High-speed users are generally a little happier about everything, with top 2 boxes averaging 79% vs. 72% for dialup.



Next we asked users to rank the importance of several enhancements we are already considering, in order to help us prioritize them.


q8.gif


The top ranked responses, in order, are:

  1. "Auto-save while writing messages so the text isn't lost if I lose my connection."

  2. "Never flag a message as spam if the sender is in my Address Book ."

  3. "Save my default font settings for writing messages."

  4. "Put common tasks in a toolbar across the top of the page."


Some others that were highly ranked by users are:

  • "Insert pictures inline in the body of a message."

  • "Allow more than 3 file attachments on outgoing messages."

  • "Allow easy switching between mailboxes while logged in."

  • "Improve searching and sorting in the Address Book."



We then asked three open-ended questions:


74% of respondents answered one or more of these, and 52% answered two or more. Here's the top 20 responses for each one. Some of the same general responses show up in each of the questions (we'll combine them in a minute).

9. Without limiting your answer to the list above, what one new Web Mail feature would be most useful to you?

RankGeneralized response
1More personal control of spam (higher block list limit, allow blocking top-level domains, allow custom filters)
2More storage space
3Auto-save while writing messages so the text isn't lost if I lose my connection
4The feature or preference already exists for what the user wants but they don't know it
5Better rich-text compose (save font settings as default, should work on Safari, make it easier to use)
6Improve the layout (make it easier, toolbar at the top, signout link at top)
7Improve the speed, particularly log-in time
8Allow easy switching or combined viewing of multiple mailboxes, bypassing login screen
9Improve address book sorting, searching, and syncing
10Be able to insert pictures in the body of new messages
11Eliminate the 3-attachment limit in Web Mail
12Remove advertisements or move them out of the way
13Improve the search feature, including message body text
14Add a positive whitelisting feature for addresses to bypass all spam filtering
15Improve reliability
16Allow compose and/or read message in separate popup window(s) so I can do more than 1 thing
17Improve the spell-check ( better dictionary, inline checking, allow add-to-dictionary)
18Make forwarding easier and more accurate, particularly with inline images and certain message types
19Support mobile devices
20(3-way tie) Login without typing domain; Larger attachment limit; Don't require login every time from same computer



10. If we could fix one thing in Web Mail that doesn't work for you today, what would it be?

RankGeneralized response
1Speed, particularly log-in time
2Advertising complaints or trouble caused by ads
3Better control of spam (higher block limit, block TLDs)
4Better rich-text compose (save font settings, support Safari)
5The feature or preference already exists for what the user wants but they don't know it
6Reliability
7Desired messages are being caught as spam
8Storage space is insufficient
9Autosave to prevent composed messages being lost; save text on lost connection
10Forwarding trouble, particularly with inline images and certain message types
113-attachment limit imposed by Web Mail interface
12User is confused about POP vs. Web Mail, can't find messages, not in sync
13Can't insert pictures inline in compose
14Address book feature/usability improvements
15Allow saving password or keep session open even after closing browser
16Allow compose and/or read message in separate popup windows so I can do more than one thing
17Allow easy switching between accounts, bypassing login
18Need a preference to return to Inbox on delete, rather than display next message automatically
19Mobile device support
20Better searching, including body text



11. Please briefly describe any other improvements you would like to see in Web Mail.

RankGeneralized response
1Remove advertisements or move them out of the way; no animated ads
2More storage space at no charge
3The feature or preference already exists for what the user wants but they don't know it
4More personal control of spam (higher block list limit, allow blocking TLDs, allow custom filters)
5Improve the speed, particularly log-in time
6Improve the layout (make it easier, toolbar at the top, signout link at top)
7Better rich-text compose with more fonts and features; save font settings as default
8Improve reliability
9Support mobile devices
10Better customer support via phone and chat
11Add a positive whitelisting feature for addresses to bypass all spam filtering
12Improve address book sorting, searching, and syncing
13Auto-save while writing messages so the text isn't lost if I lose my connection
14Improve the spell-check ( better dictionary, inline checking, allow add-to-dictionary)
15Allow larger file attachments
16Better searching, including body text
17Allow saving password or keep session open even after closing browser (so I don't have to log in every time)
18Allow compose and/or read message in separate popup window(s) so I can view more than 1 thing
19Easier forwarding
20Provide IMAP access




Now let's put it all together, combining the feature questions with the open-ended responses to come up with a top 10 list of what most users want.

Before I get to the top 10, here are a couple things worth noting. In the open-ended responses, a number of users said don’t change anything, they like it a lot just like it is.

A significant number were concerned that "changes" or "improvements" might mean making things fancier at the expense of speed and simplicity, like they’ve seen some other major email providers do. Most users want Web Mail to be fast and simple, rather than trying to imitate a desktop email program inside a web browser.

The most common comparison made to a "good" example, was to Gmail, which is full-featured but still clean and uncluttered. Only a very few users would like Web Mail to be more "like Outlook" or more "client-like" and attempt to behave like desktop software (which tends to hurt performance a lot in a browser-based application). I agree, and we aren't planning to go down that path.

Also note that in all three open-ended questions, a number of users requested a feature or setting that already exists. We released an update in April (right before the survey ran) that has a completely reorganized Preferences section, making it much easier to find the setting you are looking for. If you haven't looked at it lately, click Preferences and see what is available there.



Here's the overall top 10 user needs or concerns:

  1. Advertising - in the way; slows me down; animation interferes; I object to it in general.

  2. Speed - particularly login time; top issue for dialup users; ads perceived as a contributor too.

  3. More personal control of spam - higher block list limit; allow blocking top-level domains; allow custom filters.

  4. Storage space is insufficient.

  5. Improve the rich text (HTML) compose - save compose font settings between sessions; don't require a toggle; don't lose text when toggling; make it work on Safari; allow inserting pictures inline.

  6. Improve the layout - including a toolbar at the top with major tasks.

  7. Auto-save text during compose and allow inline re-login if session is lost so no composed text is lost.

  8. Improve overall reliability.

  9. Allow whitelisting - so that email from my contacts never goes to Known Spam.

  10. Improve the Address Book - including searching, sorting, ease-of-use, and syncing.


From the list above I can tell you that we are working on doing something about all of them, and I can make a few specific comments about some upcoming changes.

  • There are several changes coming soon that will improve speed. One involves a quicker login step due to a change in how our servers are set up, and that will happen in the next few weeks. We're also doing some things to make sure advertisements don't slow you down. Ads already load last on the page (except login page) for users of any up-to-date browser, so that the pages are ready to use before ads begin loading and you should be able to click anything you want without waiting. A few older browsers don't obey this, so update your browser if you can. We're also working hard to enforce file sizes on banners, and using a stricter standard for dialup users, so that they load quickly. And, we have a new layout coming later this year that should display most pages quicker.

  • Advertising - other than the things mentioned above, we're also about to change the main banner placement to one many users have suggested. Up until about a year ago the banner was at the top of the page, and we moved it to the right side. That is being moved back in early June. Not only is the top placement seen as a lesser evil by many users (not as in the way), it is also smaller and will load faster. And finally, the Ad-Free Web Mail option I've mentioned before is almost ready, and it will be available to subscribers in June. More details will be provided soon.

  • Items 3,5,6,7 and 10 are on our roadmap for this year and should show up in updates over the coming months. Some other requested items that didn't make the top 10 are also going to be included. We're still working on #4 (storage) but I don't have any update for now. We're working on #8 (reliability) all the time. And I'm trying to get #9 (whitelisting) implemented and should have more news on that soon.

Thanks again for your participation in the survey. You're welcome to comment here on the results or the preliminary plans I've mentioned.