OptimizationWeek.com Website performance news and views™

eGovernment Site Credibility: Comparing Speed, Accessibility, Typos, and Validity

In order to see how the US stacks up against the rest of the world online, we compared five english-based government sites for credibility. Credibility was measured using four metrics: speed, accessibility, typographic errors (see Figure 1), and (X)HTML validity. The results show that these countries fail nearly every test, except Section 508 (only India failed this test). This study and others (Mankoff 2005) show that by mandating compliance to a narrowly focused automated test, other problems invariably arise that degrade the credibility and usability of public websites.

Previous eGovernment Studies

Because they provide information to the general population including the disabled, government sites are some of the most scrutinized sites on the Internet. Even after Section 508 took effect, previous studies have found accessibility problems with government sites. As website complexity has increased, accessibility has decreased (Asakawa 2005). In a 2004 study only 22% of 50 random government home pages passed Bobby's Section 508 automated test (Ellison 2004). In a more recent study of 50 state home pages 98% did not meet the WCAG Level AA conformance, and only 70% met Level A requirements (Goette 2006).

However, automated studies give only an approximation of accessibility problems. Accessibility reviews with multiple users on screen readers received the best results among "lightweight" methods (Mankoff 2005), but still caught only 50% of known problems. Only a thorough review by an accessibility professional can give a definitive answer to a site's accessibility.

eGovernment Site Speed

To see how well the five government sites complied with the speed guidelines published in my book Speed Up Your Site (http://www.speedupyoursite.com) we analyzed government sites from Australia, Canada, the UK, the US, and India for download speed. The home pages were analyzed using our free Web Page Analyzer tool. The results of our analysis are listed in Table 1.

Summary of Results

These government sites averaged 176,174 bytes in total home page size, with 31 HTTP requests. HTML contributed 31,933 bytes, with 24.4 images adding 96,973 bytes on average to the total payload. JavaScript and CSS usage varied widely. Canada and India used no external JavaScripts while the US used nearly 19K in 2 external scripts. CSS usage varied even more widely with India embedding all of their CSS within their HTML (zero external CSS files), to the UK with over 120K of CSS in 18 files! (see Table 1) Overall, the home pages averaged 15 seconds to load useful content, and 42.4 seconds to completely load on a 56Kbps modem.

Discussion

For constituents who use a "narrowband" connection of 56Kbps or less, typically lower income or rural residents, you'd be happier in the US (King 2006). The US firstgov.gov site was the smallest site tested at 111K and loaded in 21 seconds (see Figure 2). All the other home pages took over half a minute to fully load, with Canada being the largest and slowest site tested. Ironically Canada was also the fastest site to load useful content in 6.6 seconds, due to its layered page construction. These times are beyond the attention threshold of most users. Ideally, you want to deliver something useful for your users to interact with within 1 to 2 seconds. Overall, users prefer page load times of 8 to 10 seconds without feedback, regardless of bandwidth (King 2003).

Speed Recommendations

None of these sites used HTTP compression, which would reduce the textual payload by about 75% delivered from their servers. Most of the sites use table-based layouts, which could be rewritten using CSS layout and style to save 25-50% of (X)HTML size (see Table 2). Images were large and numerous and could be reduced using CSS rollovers and styled text instead of graphic text. Behavior using JavaScript should be used to enhance the user experience, not create it. The UK in particular used overmodularized CSS with 18 external files. These files could be combined and streamlined to save HTTP requests.

Accessibility Analysis

We analyzed the above government sites for the five key elements typically included in accessible web pages as well as for Section 508 and WCAG Level 3 compliance with Bobby. We found partial support for alt attribute and link title attribute values, and all except India include a "skip to content" link (see Figure 3). Although link titles are not required (since ideally the links themselves are self-explanatory), they can be a barometer for how much attention is being paid by developers to user comprehension. None of the sites include other common accessibility elements of accesskey, and tabindex attributes (see Table 2). None of the sites complied with WCAG Level 3 (with varying degrees of errors) and all except India passed Bobby's Section 508 test. Note that Priority 1 ("must do to be accessible") and 2 ("should do to be accessible") are the most important accessibility measures, Priority 3 is going the extra mile. Only Canada had valid (X)HTML.

*Has skip to content link but no title for screen readers.
**Uses acronym element with titles and javascript to onclick urls.
~Canada, the US, and India used a table-based layout.

Suggested Accessibility Solutions

These government sites could be difficult to navigate for people with disabilities. To improve accessibility convert graphical rollover menus to text and CSS2 for higher speed and better accessibility. For the remaining images, add descriptive alt attribute values to functional images, and blank alt attribute values to non-functional images. Include accesskeys in menu options and important links with visual cues to identify the matching key combination. Our other suggested solutions can be found in our presidential homepage review as well as the references listed at the end of this study.

Typographic Error Rates

Sites that are perceived to be more credible are more likely to be believed and succeed, according to Stanford's B. J. Fogg (Fogg 2003). Fogg cites typographic errors, popup ads, errors, and slow downloads as some of the most important factors that degrade web credibility. For each government site we tested every page for typographic errors using our crawler-based spell checker in our partnership with TextTrust (see Figure 1). Four had a number of typographic errors, ranging from one every 92 pages for the UK (see Figure 4), to one every twelve pages for India (see Figure 5). Users who see error rates this high will perceive sites to be less credible and are less likely to trust the information presented.

Conclusion

In our opinion, all of the five government sites tested had credibility problems. All five sites failed our size and download time guidelines, all but one had typographic errors, all failed our accessibility tests and WCAG Level 3 (although only one failed Bobby's Section 508 test), and only one site, Canada, passed the W3C's (X)HTML validator.

The US had the fastest and smallest site with Canada and the UK the largest and slowest sites. Australia had the best accessibility score with India the worst including the most HTML errors. After our initial review Australia dramatically improved their error rate to zero and India had the highest error rate. There was no clear winner among these sites, all need some attention to boost credibility.

References

Anderson, T., "Australian University Website Validity and Accessibility Tests,"

Surveyed eGovernment sites in the UK, the US, and Australia for measurable metrics. Found that UK sites had fewer validation errors and more accessibility features than their counterparts in Australia and the US. July 27, 2007.

Asakawa, C., "What's the Web Like If You Can't See It?,"

Studied how a blind person experiences the web over time. Found that while the use of alt text and skip navigation links has improved, the actual implementation left something to be desired. Found that the use of images has increased by 450% for a small sample of web pages from 1997 to 2005. Recommended that Web developers experience how people with disabilities actually access the Web to create truly accessible and usable sites. W4A at WWW2005, 10th May 2005, Chiba, Japan. ACM 1-59593-036-1/05/05.

Dispain, G., 2006 "2006 web standards audit of Australian Government Home Pages"

Gavin Dispain audited 105 Australian government home pages for standards compliance, WAI, and speed during the month of December 2006. Gavin presents his findings in this report, including results obtained from our Web Page Analyzer desktop.

Ellison, J. Assessing the accessibility of fifty United States government Web pages: Using Bobby to check on Uncle Sam

Found that only 11 of 50 random government home pages received the "508 Approved" icon on a Bobby report. "If the U.S. government cannot ensure accessibility of its own Web pages, it cannot credibly advocate that other organizations provide accessible information." First Monday, volume 9, number 7 (July 2004).

Fogg, B.J. Stanford Guidelines for Web Credibility

Persuasive Technology Lab. Stanford University, 2002 (revised November 2003). In two surveys of over 2600 people Fogg found that a "clean, professional look" was cited by 46.1% of participants when evaluating sites for web credibility. Information Design/Structure was cited 28.5% of the time, while Information Focus was cited 25.1% of the time. While the factors varied for different types of sites, disguised advertising and popup ads, stale content, broken or uncredible links, difficult navigation, typographic errors, popup ads, and slow or unavailable sites were found to harm credibility the most.

Goette, T., Caroline C. and J. DanielsWhite, "An exploratory study of the accessibility of state government Web sites,"

This study analyzed 50 home pages of state websites for accessibility. Found that 98% did not meet the W3C - Web Accessibility Initiative Web Content Accessibility Guidelines for Conformance Level AA. Only 70% met Conformance Level A requirements, and not one state met Conformance Level AAA standards.The authors found that the majority of state home pages are trying to comply with Conformance Level A. However, 20% of the states do not have a text version or a Web accessibility policy link on their home page. Univ Access Inf. Soc (2006) 5: 41–50 DOI 10.1007/s10209-006-0023-2

Hackett, S., Parmanto, B. and X. Zeng, "Accessibility of Internet Websites through Time,"

Using the Wayback Machine, a random sample of 315 websites (including 22 government websites) from 1997 to 2002 were analyzed for accessibility and complexity over time, compared to government websites. The authors found that websites have become progressively inaccessible through the years while complexity has increased (scripts, objects, and tags). Government websites have maintained their original accessibility levels while complexity has increased. The authors conclude that increasing complexity "inadvertently contributes to increasing barriers to accessibility for persons with disabilities." ASSETS'04, October 18–20, 2004, Atlanta, Georgia, USA. ACM 1-58113-911-X/04/0010

King, A., "Response Time: Eight Seconds, Plus or Minus Two,"

in Speed Up Your Site: Web Site Optimization (http://www.speedupyoursite.com), Indianapolis: New Riders Publishing, 2003. The consensus among HCI researchers is to deliver useful content within 1 to 2 seconds (navigation bar, search form) and your entire page within 8 to 10 seconds (8.6 seconds was the figure most cited).

King, A., "The Bandwidth Report

Monthly summary of bandwidth trends in the US and elsewhere. Tracks bandwidth penetration among active Internet users and the population at large. Found that higher income and more educated households tend to have higher speed connections. Website Optimization, LLC. 2006.

Mankoff, J., Fait, H., and T. Tran, "Is Your Web Page Accessible? A Comparative Study of Methods for Assessing Web Page Accessibility for the Blind,"

Compared different methods of assessing web page accessibility appropriate for web developers without accessibility experience. Compared laboratory study with blind users to an automated tool, expert review by web designers with and without a screen reader, and remote testing by blind users. Found that multiple evaluators using a screen reader were most consistently effective at finding both empirical and WCAG accessibility problems. However even the multiple screen reader method found only 50% of the known accessibility problems. Found that "automated web accessibility testing tools and guidelines alone are inadequate for web designers with little accessibility training." CHI 2005, April 2-7, 2005, Portland, Oregon, USA. ACM 1-58113-998-5/05/0004

About the Author

Andy King is the founder of five developer-related sites, and the author of Speed Up Your Site: Web Site Optimization (http://www.speedupyoursite.com) from New Riders Publishing. He publishes the monthly Bandwidth Report, Optimization Week, and Speed Tweak of the Week. Thanks to TextTrust for their assistance.

Disclaimer

Any trademark or tradenames used in this article are owned exclusively by their owners and they do not endorse or sponsor this site. Optimization Week and the author are not affiliated with any of these sites any form, and we do not endorse these companies. All opinions expressed herein are based on information that we believe to be reasonably accurate at the time of publication. If you find any errors or misstatements of fact please contact us.

Copyright © 2003-2020 Website Optimization, LLC - Try our new conversion rate optimization service, and free website speed test Local Dumpster Rental | Sleep Supplment
Last modified: August 29, 2007