Bulletin

Stephen Smith
Webmaster, Good News Publishers (webmaster@gnpcb.org)
August 5, 2002. Updated July 10, 2006.

Introduction

The creation of the ESV Online Edition occurred in June and July 2002, driven
by our twin goals of fidelity to the printed version and ease of use. In this
document we reveal the justifications for our decisions from both usability
and technical perspectives. We hope that future implementers of Bible sites
will use this document as a baseline for development and will not have to rediscover
the lessons we learned.

Creation of XML Files

The first step in developing the Online Edition lay in creating a set of
XML files that accurately represented the text of the Bible. Working from a
basic set of XML files, we added tags as necessary to ensure the online version
reflected the printed version. Using these XML files as a base, we were able
to transform the text easily into whatever format we needed.

• DTD descriptions
  • Master XML files
  • Character entities
    only
  • Cross references
  • Footnotes
• XML-Schema descriptions
  • Master XML files
  • Basic elements included
    in both the Master XML files and the web service file
  • Format in which we make
    the files available as a web service

Comparative Usability Study

Background

In July and August 2002, after the test version of the ESV went online, we
conducted a small comparative usability study of the Bible search engines that
existed on the Internet at that time. We wanted to ensure that the ESV’s
usability equaled or exceeded that of other online Bible sites.

We followed standard usability testing methodology, including the use of
think-aloud protocol. We tested four users on the following sites:

• Bible Gateway
• Crosswalk
• New
  Living Translation
• Holman Christian Standard
  Bible
• Blue Letter Bible
• ESV Online Edition

We did not test the NET Bible because
it did not allow word searching. The order the users encountered the sites
varied. Every user was a Good News Publishers employee, so we can hardly claim
their impartiality; however, because we only wanted a general idea of the usability
issues raised by current implementations, we deemed these users sufficient.

Each user completed the following four tasks:

1. Show me 1 Timothy 3:15.
2. Show me the rest of 1 Timothy 3.
3. Where does Jesus say, “I am the light of the world?”
4. Show me the first chapter of Mark.

We considered these tasks representative of what users would want to accomplish
when visiting a Bible site: looking up a passage and locating a verse containing
specific words. We also wanted to evaluate how the sites facilitated navigation
between related tasks, such as moving from a single verse to seeing the verse
in context.

Results

Because our intent in this document is to share considerations that designers
should take into account when creating an online Bible, and not to discuss
the weaknesses of specific sites, we are publishing only our recommendations.
However, these recommendations reflect difficulties encountered by users at
one or more sites.

Recommendations (Best Practices) for the Design of a Bible Site

Input Areas

• Use one input box to handle both passage lookup (e.g., John 3:16)
  and word search (e.g., love). Users disliked having to think about
  which box they should use, and often entered queries in the wrong one.
• Do not make the user tell the site whether he or she is looking for a
  passage or searching for a word. The site should be able to figure out the
  intent in most cases and should ask the user to clarify ambiguous queries.

Handling Input

• Do not find pattern matches anywhere in words (may should not
  also find dismay). This behavior momentarily confused users, who
  were not expecting it.
• Recognize common book abbreviations, such as 1 Tim. Below,
  you can find the pattern-matching algorithm we use.
• Allow the use of Roman numerals in book names (e.g., the site should recognize II
  Timothy as meaning 2 Timothy).
• When a user enters a word-search query, search for all the words in any
  order. Do not force an exact-phrase search unless the user requests it, and
  do not search for verses containing any of the words, but
  rather all of them (i.e., use a Boolean and rather
  than or).
• Allow double quotations marks to indicate phrase searches (e.g., “in
  the beginning” should find that phrase).

Search Scope

• Use appropriate default scope. A search should by default find matches
  in the whole Bible, rather than part of it. If the whole Bible is not available,
  the search should default to as much of the Bible as possible.
• If only part of the Bible is available, indicate such and do not allow
  searching the whole Bible.
• Indicate which part of the Bible particular searches cover, especially
  if not the whole Bible. A search results page might say something like, “Found
  16 matches in the New Testament.”

Design of Passage and Search Result Pages

• Tell which translation the site is displaying. Especially important for
  sites that display multiple translations, this recommendation nevertheless
  applies to single-version sites, as well.
• Emphasize the content rather than overly elaborate navigation systems
  with multiple buttons and input areas. Users wanted to see the results of
  their queries, not worry about what to do next. Designers should treat result
  and passage pages as destinations, rather than as waypoints. They should
  emphasize the display of the content and subordinate navigation—the
  eye should not focus on the navigation unless the user wants to go elsewhere.
• Include search at the top and bottom of these pages. Users disliked having
  to go back or return to the top of the page to conduct another search. Additionally,
  they did not always see the search box if the site only showed it at the
  bottom of the page.
• Allow the user to refine the search on the Search Results page, especially
  if the query returned zero or many results. Show the user’s query in
  an input box on the Search Results page and allow the user to change the
  search from there.
• Consider allowing one-verse-per-line display in addition to paragraph
  display.
• Use a large, legible font. Users disliked having to squint to read the
  text.

Terminology

• Use unambiguous number formats. For example, do not say just “3
  of 16” when you mean “Chapter 3 of 16.”
• Do not use technical terminology relating to searches, such as scope or Boolean.
• When using abbreviations, use unambiguous and immediately recognizable
  ones. For example, Joh (John) looks a lot like Job, especially
  at smaller print sizes.

Navigation

• Indicate whether Previous and Next buttons mean “Go
  to the previous/next chapter,” “Go to the previous/next verse,” or
  something else.
• Show Previous and Next buttons at both the top and bottom
  of pages that display passages. Users disliked having to return to the top
  of the page.
• Do not have in close proximity more than one input box that performs the
  same function. For example, do not have two functionally identical search
  boxes near each other; users did not understand the difference between them
  (if one existed).

Displaying Content

• Tell which books, chapters, and, if relevant, verses the user is seeing.
  If the user is looking at only part of a chapter, show the chapter number.
• Use superscripts or a similar mechanism for footnotes. Do not underline
  words in the text to indicate footnotes, as users did not understand that
  convention.
• Use one footnote mark in the text per footnote. Users had trouble matching
  the footnotes to the text when one footnote mark indicated several footnotes.
• Show only a single verse if the user requests a verse reference (e.g., John
  7:53). Users could see the merit of showing the surrounding verses
  when searching for a single verse, but they ultimately preferred that the
  site show only what they requested.
• Let users easily look at a verse’s context by allowing them to see
  surrounding verses.

Technical Issues

• Test for and eliminate technical bugs, such as not honoring a user’s
  request to search only part of the Bible. Below, you can find some of the
  queries we used to test the ESV.
• Make the site as fast as practical.
• Have an overview page for people to link to. It’s surprising how many people link to the browse page instead of the home page.

Implementation

This part of the document explains how the site works and reveals the justifications
for our design.

Usability Perspective

Though we did not conduct the usability study until after posting a test
version of the site online, we conducted small usability tests throughout the
site’s development to ensure that the site’s structure and display
made sense to users. Following are the key points we considered and the choices
we made for the major pages in the site.

All pages

• All pages have a logo in the top-left corner of the screen that serves
  to orient the user.
• Search boxes occur at the top and, except for very short pages, bottom
  of every page to allow the user to refine the current search or go in a new
  direction. Users disliked having to use the back button or scroll to the
  top of the page. The front page lacks these search boxes because we found
  their proximity to the central search box confused users.
• Copyright and quotation permission information appear at the bottom of
  every page.

Search

• Search is case-insensitive.
• We eliminate most punctuation from a query—everything except commas
  when they occur in numbers, possessive apostrophes (to stay consistent with
  how the printed ESV concordance handles them) and colons to indicate chapters.
  In verse references, we convert semicolons and periods to commas and colons,
  respectively.
• Search handles both passage lookups and word searches, though not at the
  same time.

Error Messages

• Error messages occur in red at the top of the page. Unfortunately, tests
  of our site indicate that users still ignore them unless they feel confused.
• The wording of the messages tries to give constructive feedback and offer
  the user steps to take to address the issue that caused the error.
• For certain known searches, such as 1
  Maccabees, we display a specialized error message. Below,
  you can find all the searches we handle this way.

Empty Search Results Page

• We try to display a possible spelling correction when the query returns
  no results.
• Originally, the Advanced Search form appeared on this page in case the
  user wanted to take advantage of it. Our tests indicate that users appreciated
  this behavior; however, a review of our search logs revealed that users found
  it confusing. Thus, we present a new recommendation: only show the Advanced
  Search when users specifically request it.

“Too Many Results” Search Results Page

• The heading says, “Found over x results.”
• A dropdown menu to limit the search to part of the Bible appears so the
  user can edit the query’s scope directly from the results page.

Normal Word-Search Results Page

• The heading tells the number of results found and repeats the query.
• The search boxes appear filled-in with the user’s query to allow for easy
  editing.
• Multiple results pages are identified by a passage range (e.g., Genesis
  1:22-Exodus 17:3) rather than page numbers to allow the user to
  quickly find the relevant part of the Bible.
• When a query finds over five pages of results, the list of other pages
  becomes a table with columns for easier scanning.
• A scrolling box may appear when a query finds over eight pages of results.
  This solution was the subject of much discussion; it was a compromise between
  displaying only Previous/Next buttons and all the result
  pages. The former provided too much rigidity, while the latter often pushed
  the beginning of the search results off the portion of the page visible without
  scrolling, leading to confusion.
• Footnotes in search results appear on a new line immediately below the
  verse. Users preferred this approach to displaying footnotes directly in
  the text.
• Headings of various types appear on a new line and in a different font,
  allowing easy identification. Again, users preferred this approach to inline
  display.
• The query terms are highlighted in several different colors to allow fast
  differentiation.

Advanced Search Page

• Clicking the link to Advanced search fills in parts of the form
  based on the current settings.
• Users do not always realize that the Show this passage and Advanced
  search buttons do not perform the same action. We do not know how
  to make this distinction clearer.

Passage Display Page

• Footnotes appear at the bottom of the page. Clicking a footnote link brings
  the user to that footnote. Clicking the footnote link at the bottom of the
  page returns the user to the place in the text where the footnote occurs.
• Hovering over a footnote link in the text displays the text of the footnote.
  Users have trouble discovering this behavior, suggesting that it may need
  to be more prominent.
• Footnotes at the bottom of the page show the verse reference in which
  they appear (e.g., 1:15), providing further confirmation to the user that
  he or she is looking at the correct footnote.
• Previous/Next buttons appear at the top of each passage displayed and
  at the bottom, too, if only one passage appears.
• At the bottom of the page, users can use a form to compare the passage
  to another translation (via the Bible Gateway).

Coding Perspective

We wanted our code to be modular and interconnected, so a change in one location
would not require changes in many locations.

Web Standards

Because we stored the text of the Bible in XML, we decided to code the site
in XHTML 1.0 Transitional. We tried to avoid using presentational markup and
tables for layout as much as possible, but in some cases browser limitations
made them more attractive than strictly structural solutions. (At the time
of this writing, most people are using Internet Explorer 6 and 5.) Dealing
with browser quirks in displaying CSS probably added a couple of days to the
development cycle, but using XHTML will save us time in the long run when we
do not have to rip out our table-based layout and replace it with more logical
markup. In time we hope to remove all presentational markup.

A benefit of using XHTML is its ability to display on a number of devices
(such as mobile phones) without large changes. As mobile devices become a common
means of accessing the Internet, this benefit will become increasingly important.
We also believe that a site’s accessibility (for partially sighted users,
for example) is important, and using logical markup facilitates accessibility.

We hid CSS stylesheets from Netscape 4 and other outdated browsers. They
receive a plain but usable version of the site.

Our other reason in using stylesheets stemmed from a desire to avoid having
a separate “printer-friendly” version of each page. All pages on
the site use a print stylesheet that omits much web-specific navigation and
changes fonts and spacing to make them better suited to print. (For reference,
please see http://www.alistapart.com/stories/goingtoprint/.)

Database Schema

The MySQL database that powers the site has four tables, as described below.

The column unit_id is a zero-padded eight-digit number that
uniquely identifies each verse in the Bible: The first two digits are the book
number (01-66), the next three digits are the chapter number (001-150), and
the last three digits are the verse number (001-176). Using this setup allows
us to parse the string to determine which verse we are seeing, rather than
having to go to the database and retrieve the related values. For example,
John 3:16 has a unit_id of 43003016.

CREATE TABLE esv_xml (
    book tinyint(3) unsigned NOT NULL default '0',
    chapter tinyint(3) unsigned NOT NULL default '0',
    verse tinyint(3) unsigned NOT NULL default '0',
    unit_id int(8) unsigned zerofill NOT NULL default '00000000',
    begin_par char(1) default NULL,
    begin_indent char(1) default NULL,
    end_par char(1) default NULL,
    end_indent char(1) default NULL,
    content text NOT NULL,
    PRIMARY KEY (unit_id),
    KEY book (book,chapter,verse),
    KEY book_2 (book,unit_id)
    )

CREATE TABLE esv_text (
    unit_id int(8) unsigned zerofill NOT NULL default '00000000',
    header varchar(255) NOT NULL default '',
    content text NOT NULL,
    footnote text NOT NULL,
    PRIMARY KEY (unit_id),
    FULLTEXT KEY header (header,content,footnote),
    FULLTEXT KEY content (content),
    FULLTEXT KEY footnote (footnote)
    )

CREATE TABLE esv_words (
    word varchar(20) NOT NULL default '',
    phonetic varchar(10) default NULL,
    occurrences smallint(5) unsigned NOT NULL default '0',
    PRIMARY KEY (word),
    KEY phonetic (phonetic)
    )

CREATE TABLE esv_books (
    book tinyint(4) NOT NULL default '0',
    full_name varchar(15) NOT NULL default '',
    short_name varchar(6) NOT NULL default '',
    max_chapter tinyint(3) unsigned NOT NULL default '0',
    testament char(3) NOT NULL default '',
    section varchar(17) NOT NULL default ''
    )

PHP Backend

We coded the site in PHP 4.2+, using Dreamweaver MX as our coding environment.

Available Source Code

• See the source code for the file that turns a given query into an easily manipulated object.
  Please note that this file does not validate the returned object to see
  whether the given passages actually exist. Also note the copyright restrictions
  contained in the file.
• See the source code for the file that contains the known abbreviations for Bible books. The first column is the book number, and subsequent columns
  are Perl-style regular expressions describing acceptable variations. You
  are free to download this file for any use you wish, though credit is appreciated.
• See the source code for the file that silently corrects common spelling errors. The first column is the spelling error (again in Perl regular-expression format),
  and the second is the correct spelling. You are free to download this file
  for any use you wish.
• See the source code for the file that contains queries we handle specially, by giving an error message. The first column is the term searched for (in
  Perl regular-expression format), and the second is a code telling us which
  message to display. You are free to download this file for any use you
  wish.

Handling Input

The parsing engine needs to handle all of the following queries intelligently.
While some of them fall into the realm of “corner cases,” others
happen frequently. We believe our solutions accord with user expectations,
but you may wish to handle certain cases differently.

Other Features

Since the original writing of this article, we have implemented:

• A Devotions area based on two tracts we
  publish, a book we publish, and The One Year® Bible. (December 2002 and May 2003)
• A Web Service that allows web developers
  to use the text of the ESV directly on their websites. (February 2003)
• Links to an audio version of the New Testament when looking at a New Testament passage. (May 2003)
• Several RSS feeds, including a verse of the day and passage of the day (drawn from the Devotions area). (July 2003)

Future Directions

We may allow the saving of certain settings, such as “Show me red-letter
text” and “Show me one verse per line, rather than in paragraphs.”

Bible Book Query Patterns

The table below shows the distribution pattern of inputs and abbreviations for all the books of the Bible, based on approximately 500,000 queries. In developing a Bible site, you should probably support any queries that account for over 1% of the queries for a given book. Known misspellings were converted to the closest form (e.g., Galations became part of the total for Galatians). If you’re interested, download this Excel spreadsheet (44 KB) to see different ways of looking at the data.