"http://www.i-a-l.co.uk/CIQM/QualCrit.htm"
Vgl.: Information Automation Limited and CIQM, 1999

Database Quality Criteria

The following points are largely taken from Reva Basch's article in October 1990 Database Searcher, Measuring the Quality of the Data: Report on the Fourth Annual SCOUG Retreat.
SCOUG (the Southern California Online User Group) came up with ten headings or criteria by which databases should be judged; while I do not always agree with the way in which they have divided topics by these headings ("currency" under the heading "accuracy", for instance), they do make a useful guide. The headings are: Consistency; Coverage/scope; Error rate/Accuracy; Output; Customer support and training; Accessibility/Ease of use; Timeliness; Integration; Documentation; Value to cost ratio.

Consistency
This is interpreted to mean the extent to which records within a database follow the rules with regard to field assigments, field tags and other data elements, as well as indexing and editorial decisions.

Coverage/Scope

How well does the database cover its subject area(s)? Is it the "authoritative" file in its discipline?
Are periodicals indexed cover to cover? If not, is the producer's policy on inclusions and exclusions clearly stated?
Are there any serious gaps (including short- or long-term gaps caused by technical problems) or misinterpretations? Are temporary problems or changes in file content flagged in a log-on banner?

Timeliness

How often is the file updated?
How current is the material included in each update?
Does one type of publication have priority over others? If so, is this documented?
Are time-sensitive sources available in near-real-time?
Does the load cycle vary from system to system(host to host or medium to medium), so that one version of a database is consistently more or less timely than another?

Accuracy/Error rate

What data sources are used? How current and complete are they?
Is there a prevalence of typographical errors, incomplete records or other dirty data? What percentage of errors occur in critical fields (eg company name, numeric)? What is the error rate per update?
What percentage of records are received by the database producer on tape as opposed to being keyed or scanned in?
What quality control procedures does the producer employ?
Are faulty records identified and/or removed and corrected?
Are corrections in the source material or print equivalent incorporated in the database? If so, how promptly?
Is there a mechanism for users to tag faulty records and bring them to the attention of the database producer?
Are searchers compensated for unusable information?

Accessibility/Ease of use
This category covers both access to online services and, at the database level, access to the information itself.

Does the system require special software or dedicated hardware?
Can the user sign up online, or must written contracts be negotiated first?
Is the database accessible through gateways as well as directly?
Does the system support full and variable proximity searching? Are there limits on word adjacency?
Is there a mechanism for searching literals and stop words as parts of phrases?
Are there built-in equivalences and on/off automatic pluralization?
Can terms be plucked from an online thesaurus and used in a search without re-keying?
Can a search strategy be saved and re-used in another database?
Is there a multilingual thesaurus with an automatic synonym option?
Within the database itself, how deeply and flexibly are the records indexed? What kind of subject indexing, if any, is used?
Which data elements are searchable? If some are display-only, is this clearly documented?
In full-text records does a "KWIC" display blow back to the current page rather than to the beginning of the record? [blow back is their phrase - I leave you to guess!]
Is there a "locate" feature that will display the occurrence of any term - not just search terms - in an existing answer set?
In there a well-maintained online thesaurus?

Integration

Does the overall database structure resemble that of other files of the same type that are likely to be searched with it?
Do field and output format tags match those used in comparable databases on the same system?
Can multiple files be searched together?
Does the system support duplicate detection and removal?
Are there system-wide journal lists and other cross-database search aids?
Have links been forged between bibliographic and full-text records in the same database and in different ones?
Can one link from textual records to related information in external, perhaps multimedia sources?
Can one search for a journal or other source by name, without knowing or specifying the database that carries it?

Output

Can the user define custom formats? Can pages or portions of a document be printed selectively?
Can search results be batch-printed and sent by mail, electronic mail, diskette or fax?
Can print commands be bundled or stacked?
Can search output be downloaded into spreadsheets or database management programs?
Is downloading possible with the correct formatting (eg for tables)?
Is there an auto-logoff option that will disconnect from the system at the end of a print or download session?
Does the system support data compression? Error checking?
What is the highest baud rate that the system supports?
Are record aesthetically acceptable, eg not all in upper case?
Are tables and graphic material included, or just referenced? If included, how are they formatted and presented? If not, are captions included?

Documentation

Is both print and online documentation timely, accurate and readable? Is it available free or at nominal cost?
Is a print thesaurus available?
Are there regular mailings of newsletters, search aids and other support material? Is this information practical and useful?
Is documentation locally-resident (as opposed to online), well designed and easily searchable?
Is online help available? Is it context-sensitive?
Do log-on banners and other online pointers alert the user to database limits and useful features, as well as to temporary problems?
At the source level, does database documentation outline editorial policies with regard to coverage dates, currency, inclusions and exclusions?

Customer support and training

Is there a toll-free number for customer service?
Is live, knowledgeable customer support available whenever the system itself is up?
Can customer support be reached via electronic mail, and is it responsive in this fashion?
What kind of training is available, both basic and advanced?
Is training free or reasonably priced? Do training sessions include free post-session practice time?
To what degree is training available outside the main urban centres?
Is customer support available in countries other than that in which the system or database producer is located?
Are there computer-based or video tutorials, or other forms of self-paced instruction?
Is there a low-cost or after-hours practice file?
Does the producer or online service support user advisory panels, etc?
Is free time offered on a file being used for the first time? Is there a "free file of the month" policy?
Are changes made to database without warning/documentation for users?

Value-to-cost ratio

Do general system characteristics like speed and performance, pricing structures and display options support efficient, cost-effective searching?
Is there at least one free or inexpensive default format? For documents, does this format include title and publication year? In hybrid files, does it include a full-text indicator?
Is a KWIC display supported?
Are display charges for various formats consistent with other databases? How well do these costs reflect the value of the data elements? How are user-defined formats priced relative to fixed formats that contain the same information?
Does it cost about the same to print offline as it does to print online (assuming the connect charges associated with the latter will about equal the "per hit" cost differential between an online and a more expensive offline print)?
Can search results be sorted or relevance ranked?
Does the break key work while the system is processing a search, as well as during document display? Is there a charge for items included in the display command that have not yet been displayed? Is there a charge for partially displayed items?
Are multiple hit charges imposed every time the same item is displayed in a search session, or do costs "top out" at a one-time charge for the fullest format in which the item was displayed?
Is cost accounting available at any point during a search session, as well as at log-off?
Are there up-front subscription fees, monthly maintenance costs or minimum usage charges?

In addition to these general measures, SCOUG came up with some criteria specific to the type of database.
Bibliographic files should offer the following:

Ability to restrict by geography, language, subject facet
Indexing from top to bottom of subject hierarchy
Treatment (eg policy, review, theoretical)
Links to related records, eg entire conference and individual papers presented
Exact author and title as well as equivalents
Journals and sources spelled out in full with no abbreviations
Fully-searchable author affiliations
All data elements searchable, separately tagged and individually displayable
Minimum required fields: author, affiliation, title, source, country of origin, publication date, abstract, unique numbers (registry, contract, etc), indexing
Standard format for author entry across databases
Same file structure for similar databases, with standardized field labels
Hierarchically-organized online thesauri
Online and manual look-up tables for chemical codes, etc.

Full-text databases have these special needs:

Fully-searchable records, with field searching possible as well
Low-cost browsing format should include document length
On/off toggles for automatic pluralization, equivalencies, synonyms, etc
Comprehensive coverage at the source level. Exceptions, gaps and cessations should be documented
Strongly subject-oriented files should include the major publications in the field
Quick loads, with currency equivalent to that of print sources
KWIC, highlight, continuous scrolling

Directory databases come in many forms - company, product, association, biographical, etc. Questions of content and access are especially crucial here, because a directory file is used like a reference book. If a listing is missing, inaccessible or flawed, there is often nowhere else to look for it.

Include basic contact/address information regardless of directory type
Hierarchical geographic searching from zip code to regional level
Indicate source and date of information, identify "dead" companies or remove them from the file
Company financials should specify units of currency and, for non-US sources, when currency was revaluated
Links among parent companies and subsidiaries
For product directories, include price, product description and/or evaluation, purchasing contact. Use a standardized parts nomenclature, eg SIC code, patent number, company stock number
Error correction protocols to ensure accurate downloads of numeric data
Mailing label output where appropriate

© Information Automation Limited and CIQM, 1999
Links checked: 13 January 1999 14:25:18
This page last updated 13 January 1999 14:25:23
Pages maintained by IAL