1. Objectives
2. What is a Literature Review?
3. Purposes Served by a Literature Review
4. Steps in a Literature Review

1. Choosing Your Topic/Formulating Your Question(s)
2. Selecting Where to Search (Databases, Indices, Catalogs)
3. Mapping Your Terms (Identifying Key Words/Descriptors)
4. Combining the Mapped Key Words/Descriptors (Boolean Logic)
5. Modifying Your Search Strategy (Narrowing, Expanding, Limiting)
6. Selecting, Retrieving and Storing Your Source Documents
7. Abstracting and Evaluating Your Source Documents
8. Organizing and Writing the Review

Objectives

In this unit, you will learn:

• What purposes a literature review serves
• What steps are involved in reviewing the literature
• How to search for and locate relevant titles, abstracts, and source documents
• How to evaluate, select, and abstract pertinent literature
• How to organize a written literature review

What is a Literature Review?

A literature review (or overview) is a summary and analysis of current knowledge about a particular topic or area of inquiry. There are two major types of published reviews: non-systematic (or journalistic-style) reviews and systematic reviews.

Journalistic-style reviews (also called overviews) are currently the most common type. A journalistic review provide a summary of evidence derived from mostly primary studies that have been selected and synthesized according to the author's personal and professional perspective (somewhat like a book chapter that focuses on a specific topic). Journalistic reviews can cover a wide range of subject matter at various levels of completeness and comprehensiveness.

Systematic reviews, on the other hand, summarize and analyze related primary studies that have been searched for, evaluated, and selected and reported according to a rigorous and predefined methodology. Specifically, a systematic review:

• Focuses primarily on the results of high-level evidence, e.g., randomized controlled clinical trials (RCTs)
• Provides an explicit statement of objectives, materials, and methods
• Has been conducted according to an explicit and reproducible methodology
• may include synthesis and analysis of pooled data across several studies (meta-analysis)

The following diagram outlines the basic methodology used in conducting systematic reviews of the literature:

Figure from: Greenhalgh, T. (1997). How to read a paper Papers that summarise other papers (systematic reviews and meta-analyses). BMJ, 315, 672-675.

Perhaps the best way to understand the difference between these two types of reviews is to read examples of each. The following table presents 5 examples of each type of review. To access any example, simple click its title. Please note that these examples include only the textual content.

What type of literature review you conduct depend on your purpose and your audience.

Purposes Served by a Literature Review

The type of literature review you conduct depend on your purpose and your audience. Literature reviews serve many purposes and audiences. The two primary 'users' of literatures reviews are researchers and clinicians.

For Researchers

All good research includes a literature review (often included in the Introduction or Background section of a journal article). Research begins with ideas and concepts that are related to one another through questions or hypotheses (anticipated relationships). Although these ideas and concepts may simply 'pop into the researcher's head,' most often they derive from a careful exploration of the collective body of prior related literature. For this reason, literature reviews conducted by researchers tend to be exploratory in nature and thus more journalistic in style. However, even a research-oriented journalistic style review should provide readers with explicit information on the sources of evidence used and the relative weights given to these sources.

For researchers, a review of the literature serves three key purposes:

• It helps uncover ideas about which variables are important in a given field of study
• It provides information about what research has already been done and what needs to be done
• It reveals demonstrated and/or theorized relationships among variables that the researcher is interested in or wishes to study

Discovering Important Variables

A review of the literature helps researchers identify and select pertinent variables for study and also helps determine existing or conceptual relationships among the selected variables.

What Has Been Done/What Needs to Be Done

Of course, researchers need to review past work to avoid repeating it. More important however, is the knowledge that prior research serves as a springboard for subsequent efforts -- new studies build upon and extend the old. Many studies, for example, conclude with a section on suggestions for further research. By carefully examining major studies in a given field (and their suggestions for further study), a researcher can identify which directions are worth pursuing and which are not.

Determining Meanings and Relationships

To conduct meaningful research, prior relationships among variables in the chosen area must be explored, examined, and reviewed. Only in this manner can the researcher build both the context and case for conducting a pertinent investigation. Such a review not only helps the researcher better understand the phenomena under study, but also can provide meaning and context to readers.

For Clinicians/Practitioners

For clinicians and practitioners, conducting a literature review can help summarizes past work in a field and thus provide an up-to-date perspective on a particular aspect of health care delivery. Often, a good literature review yields useful conclusions about how best to apply diagnostic or therapeutic methods in day-to-day practice (this is why good literature reviews are a component of all quality practice guidelines and protocols) . A good literature review also helps distill the enormous and growing body of knowledge characterizing most health care fields down a to a smaller number of workable conclusions that can then be made available to researchers and practitioners alike. In general, the review of the literature conducted for clinicians and practitioners should be more systematic in nature, focusing on answering explicit questions of safety or efficacy, i.e., does this therapy work on these patients?

Steps in a Literature Review

To conduct a literature review, you generally proceed through eight key steps or phases: (1) choosing your topic and formulating your question(s); (2) selecting where to search (databases, indices, catalogs); (3) mapping your terms (identifying key words/descriptors); (4) combining the mapped key words/descriptors; (5) modifying your search strategy (expanding, narrowing, limiting as needed); (6) selecting, retrieving and storing your source documents; (7) abstracting and evaluating your source documents; and (8) organizing and writing your review.

Step 1 - Choosing Your Topic/Formulating Your Question(s)

A literature review requires direction and focus. The more general it is the less useful will be its outcome. Therefore, your first step should be to identify an area or topic that interests you.

Identifying an interest area is a highly personal process, but should derive from some combination of curiosity, real-world experience and knowledge of existing practices in your field. In general, the topic of a literature review should evolve out of practical questions such as "does aspirin decrease risk of colon cancer in men?" or "do chest x-rays affect the outcomes in ICU?"

In formulating your question(s), you should always try to narrow the scope sufficiently enough to assure that the review is feasible yet provides an adequate depth of coverage for the intended audience and purpose. For many, the best approach to narrowing the question is to use the PICO format, i.e.,

IDST6400 (Evidence-Based Literature Review) provides a useful worksheet for constructing your question in the PICO format, available online at:  

http://www.umdnj.edu/idsweb/idst6400/lit_review_question.doc

Always begin off-line with analysis of your question, its component concepts and terms. You should start by making a list of synonyms describing the concepts, methods or procedures (or PICO components) of interest. Just developing this list of terms can help you narrow and sharpen the focus of your literature review.

Take, for example, the PICO-refined question "do daily chest x-rays affect the the morbidity, mortality or length of stay of mechanically ventilated adult patients in ICU?" You should start by breaking the question down into its its component concepts, terms and related variables (with synonyms):

Step 2 - Selecting Where to Search

Once you have 'dissected' your question, it is time to select where to search. Given the scope of source materials commonly available in most fields, it is important that your first establish a priority scheme for your search. This priority scheme addresses the question "What types of source materials should I rely on most?"

In general, there are two major types of source material that underlie literature reviews: (1) original research (primary) studies, and (2) secondary or integrative studies. Original studies report research first-hand. Secondary studies summarize and draw conclusions from primary studies. For more detail on the different types of published studies and the levels of evidence they represent, see the Types and Levels of Knowledge section of Seeking Answers: Questions Heath Professionals Ask and the Evidence Needed to Answer Them.

A comprehensive review of the literature should be based on a search of both types of material, with the initial effort being directed at evidence-based resources.

Evidence-Based Resources

Sometimes a definitive answer to your question already exists in the literature. If this is the case, you certainly don't want to reinvent the wheel and conduct a literature review when your question is already answered. To determine if a definitive answer to your question already exists, you should first search the evidence-based literature.

Evidence-based resources are "pre-filtered" to include only those studies (primary or secondary) that meet pre-established standards of quality. Evidence-based resources include both 1) databases and compendiums of clinical trials, systematic reviews, meta-analyses and practice guidelines and 2) specialty journals devoted to disseminating evidence-based information. For a comprehensive listing and description of evidence-based resources go to:

http://www.umdnj.edu/idsweb/shared/eb_resources.htm

The premier collection of evidence-based literature databases is provided by the Cochrane Library.

"The Cochrane Library contains high-quality, independent evidence to inform healthcare decision-making. It includes reliable evidence from Cochrane and other systematic reviews, clinical trials, and more. Cochrane reviews bring you the combined results of the world's best medical research studies, and are recognized as the gold standard in evidence-based health care." (Cochrane Library home page, © Wiley InterScience)

The Cochrane Library is accessible to UMDNJ students via the Library EZProxy system at:

http://www.umdnj.edu/cgi-bin/libjournal/1731

The Cochrane Library provides searchable access to the following three key evidence-based resources:

• The Cochrane Database of Systematic Reviews (Cochrane Reviews)
• The Database of Abstracts of Reviews of Effects (Other Reviews)
• The Cochrane Central Register of Controlled Trials (Clinical Trials)

To search for practice guidelines go to the National Guideline Clearinghouse at http://www.guidelines.gov . Alternatively, you can try SUMSearch, a unique metasearch engine that gathers evidence-based clinical information from MEDLINE, DARE and the National Guideline Clearinghouse. SUMSearch as available at: http://sumsearch.uthscsa.edu/ .

If a search of the evidence-based literature indicates that your question has not been answered, your next step is to search the original (primary) research literature.

Primary Source Materials

Common Primary Sources. The common primary sources of knowledge used in formulating a review of the literature are original research articles, graduate theses and dissertations, conference abstracts and research reports.

Most experts would agree that peer-reviewed original research articles appearing in reputable scientific journals should form the core of any good literature review. This is because original research represents a field's primary source of knowledge. For details on the types of original or primary research commonly seen in the journal literature go to:

http://www.umdnj.edu/idsweb/idst6400/seeking_answers.htm#primary

Graduate theses and dissertations are indexed in Dissertation Abstracts International (hardcopy) and can be search online at most research libraries, including the UMDNJ libraries via OVID. Theses and dissertations undergo a form of peer review (a faculty committee), but seldom with the rigor involved in journal publication. The result is a huge variation in quality, even within the same institution or department. The most valuable information derived from theses and dissertations is often their extensive literature reviews.

Conference abstracts represent a 'window' on current research (just completed or in progress), often prior to subsequent journal publication. Unfortunately, conference abstracts are generally not indexed, and may or may not undergo rigorous peer reviewed.

Research reports are generated by both public agencies (e.g., NIH) and private corporations (e.g., pharmaceutical companies), and are the most difficult of all sources to locate and retrieve.

Finding Primary Source Material. While primary source documents (journal article titles) can be located via a manual search using printed cumulative indices (or by searching through yearly journal issue indices), this process is slow and tedious. For this reason, you should initially conduct a computerized search of relevant citation databases. Depending on your topic and its coverage via citation databases, your computerized search may need to be supplemented with a manual search of traditional cumulative indices. In any event, the efficiency of you search will vary directly with your knowledge of the specific database being accessed. For this reason, the time you spend in learning the characteristics of each database you plan to search will more than payoff in time saved sifting through hundreds of unwanted or unrelated titles.

There are hundreds of citation databases available for searching. To select which ones to search, you should 'match' you chosen topic to the content areas and journals covered by each available database. Click here for descriptions of the common research databases used most often by health-related professionals. Click here for a complete listing of research databases available to UMDNJ users.

For example, if your topic were primarily related to medical science or dentistry, you would likely start your search with the National Library of Medicine's (NLM) Medline database. On the other hand, if your interests were primarily in the nursing and allied health area, you would choose to begin with the Cumulative Index to Nursing & Allied Health Literature(CINAHL). If your research area primarily involved health care administration or policy issues (nonclinical), you would go first to the HealthStar database (National Library of Medicine and the American Hospital Association). On the other hand, were you doing educational research, the starting point would be the database maintained by the National Library of Education's Educational Resources Information Center (ERIC). If your topical area were to include psychology, you would access the PsycInfo database or its clinical subset database ClinPSYC.

Dealing with Indexing Time Lags. It takes anywhere from one to six months for a published journal article to appear in a citation database. This is a serious problem when conducting searches in fields or areas subject to rapid change. Fortunately, there is a source designed specifically to overcome this problem, Current Contents. Current Contents consists of three weekly series: Clinical Medicine; Life Sciences; and Physical, Chemical, and Earth Sciences. Each weekly has three major sections. The first section is a copy of the just-published contents pages from hundreds of journals and multi-authored books. The second section is the Word Index, which lists in alphabetical order all of the key words contained in the titles of the articles listed in the contents section. The third section is the Author Index for all articles appearing in the contents section. Current Contents is intended to be a quick way to access the most current literature, before it appears in citation databases. Note however that coverage is selective and most allied health journals are not included.

Secondary (Integrative) Source Material

Common Secondary Sources. Secondary sources of knowledge do not involve original research, but usually serve to summarize or synthesize existing primary sources. Secondary or integrative studies found in professional journals (in relative order of the quality of evidence they provide) include systematic reviews and meta-analyses, decision analyses, practice guidelines, consensus statements and journalistic reviews/overviews. Outside the journal literature, books and book chapters are the most common secondary source materials. For details on the secondary sources typically found in the journal literature, go to:

http://www.umdnj.edu/idsweb/idst6400/seeking_answers.htm#secondary

In general, published literature reviews are a better source of secondary knowledge than books and book chapters. Most fields provide a relatively timely source of quality secondary literature via regularly published review journals (e.g., Annual Review of Information Science and Technology, Annual Review of Neuroscience, Annual Review of Nutrition, Annual Review of Pharmacology and Toxicology, Annual Review of Physiology. Typically each article in these review journals is written by a different expert who writes an integrated summary and analysis of the current literature on a given topic in that area.

Better still are the growing number of journals devoted to publishing systematic reviews. The largest collection of systematic reviews is maintained and regularly updated by the The Cochrane Library. Generally, systematic reviews and meta-analyses should be your first choice when seeking the best evidence on a clinical topic or question.

Of course, finding a literature review that explicitly addresses your selected topic can save a lot of time, especially if you are reviewing the literature as a basis for conducting a research project. Unless it is a very recent systematic review, however, you should be careful in relying primarily on such 'finds.' First, with the exception of the Cochrane Reviews, most literature reviews become obsolete with time (how fast this occurs depends on the 'half-life' of knowledge in your chosen topical area). Second, if you depend solely on a journalistic review as your source, you are depending solely on the personal and professional perspective of that review's author(s). The result can be either a biased or limited (less than comprehensive) perspective.

Finding Secondary Source Material. You first step in finding good secondary material should be to search the specialized collections of evidence-based secondary sources, such as the The Cochrane Library. For a comprehensive listing and description of current evidence-based resources go to:

http://www.umdnj.edu/idsweb/shared/eb_resources.htm

To access review articles published in regular journals, you search the same citation database(s) used for locating primary sources. Many databases provide a way to search specifically for review literature. For example, the National Library of Medicine provides a PubMed Clinical Queries page which you can use to search just for systematic reviews on your topic. Alternatively, you can search the 'Publication Type' field to retrieve only review articles.

Access to other secondary source documents, particularly reference books and review annuals, generally starts at the library's 'Card Catalog.' Card catalogs are the heart of a library and the most important single entry point into a library's holdings. There is at least one card for every resource in the library, including books, journals (but not individual journal articles), microfilm, and A/V media. In most libraries there are three separate sets of cards, one for authors, one for subjects, and one for titles. In most research libraries, the "card catalog" is now on computer, using the same three categories. Each category is arranged in alphabetical order. Subject headings follow the indexing system used by the particular library, i.e., the National Library of Medicine Classification System (in most medical libraries), and either the Library of Congress or Dewey decimal system (most general libraries). If in doubt, ask for assistance from one of the librarians. The UMDNJ catalog is accessible online at:

http://voyager.umdnj.edu/

If you already have a topic and list of key terms in mind, you will find the subject heading category most useful. When you have identified an interesting-looking title under your desired subject heading, go to the shelves to find that book. Look also at other books on the same shelf and other books with the same general call number. Other books with the same call number will be in the same general subject area and can often provide you with a whole shelf full of references. The references and bibliographies in the books you have located will alert you to other secondary sources and many primary sources upon which the textbook material is based. When you write down a reference that you find in a textbook, it is helpful also to note where (e.g., page number) you found the reference in case you make some mistake in transcribing and need to go back to your source.

Unfortunately, library card catalogs only include documents in that library's collection. To find ALL titles covering your subject, you must refer to reference indices such as Bowker's Books in Print. Books in Print comprises three huge volumes-one for authors, one for subjects, one for titles. These list all of the books currently available from publishers in the United States. These are most useful when you have a broad topic to search, if you only have the name of an author or only the title of a book, or if you have discovered an author whose books you like and want to know what else she has written.

What About the Internet?

The Internet is the world's largest and fastest growing source of information. Unfortunately, as the Internet grows, finding pertinent and high quality information becomes more and more difficult. As a resource for serious research, the Internet has been compared to a massive library with hundreds of millions of free-standing documents (including personal pages, hate literature, and pornography) strewn in random piles throughout its stacks, with neither a catalog nor librarian to guide you.

Can one find useful information on the Internet? The answer is yes. However, to do so you must become expert in Internet searching and be willing to spend the extra time needed to sift through and evaluate the quality of thousands of pages of documents. For a good tutorial on searching the Internet, click here.

Currently, the best use of the Internet is as a portal to public or commercially-maintained primary source citation databases, such as the National Library of Medicine's free Medline search site.

Step 3 - 'Mapping' Your Terms (Identifying Key Words/Descriptors)

Most literature indices and databases are structured using controlled vocabularies of terms that may or may not correspond to those on your list. For this reason, the first step in your search should be to compare your initial list of terms to those provided by the database(s) you have selected. This process is called mapping.

Usually, each literature index or database has its own controlled vocabulary. For books, the controlled vocabulary derives from the topical cataloging system used by each library (NLM Classification, Dewey Decimal System, Library of Congress numbers, etc). For journal articles, each topical or field database has an accompanying index of terms.

For example, all of the 12+ million journal citations in the Medline medical literature database are indexed using Medical Subject Headings (MeSH for short). MeSH terminology provides a consistent way to retrieve information that may use different words for the same concepts. Actually, MeSH consists of four related publications.

1. The MeSH Supplement to Index Medicus® (a regularly published alphabetical and hierarchical listing of terms)
2. The MeSH Annotated Alphabetic List (an alphabetic listing includes notes for indexers, catalogers, and searchers, as well as historical information for assistance in searching the earlier online files).
3. The MeSH Tree Structures (a hierarchical listing of the MeSH vocabulary)
4. The Permuted MeSH (a listing of each significant word or root appearing in any MeSH heading or printed cross-reference)

To emphasize the importance of using key terms or descriptors that match the database or index you will search, let's look again at our practical example (do daily chest x-rays improve outcomes in intensive care patients?). Were you simply to do a quick computer search using "chest x-rays" and "ICU" as your terms, you would retrieve over 25,000 citations. Were you to have the time to peruse all of these citations, you would be disappointed to find only a few references directly pertinent to your topic.

Had you first searched for the applicable MeSH terms in the MeSH Supplement or Tree Structures, the reason for your disappointment would be clear. "Chest x-rays" is not a MeSH term. "X-rays" is a key term, but applies only to the physics of ionizing radiation. Using one or more of the tools listed above, you would learn that "Chest x-rays" translates in MeSH to "Radiography, Thoracic" and is located in the Diagnostic Techniques and Procedures branch of the MeSH tree structure (below):

Diagnosis [E01]

  Diagnostic Techniques and Procedures [E01.370]

    Diagnostic Imaging [E01.370.350]

     Radiography [E01.370.350.700]

Angiography [E01.370.350.700.060] +

Fluoroscopy [E01.370.350.700.225] +

Mammography [E01.370.350.700.500] +

Neuroradiography [E01.370.350.700.560] +

Radiography, Abdominal [E01.370.350.700.715] +

Radiography, Dental [E01.370.350.700.720] +

Radiography, Interventional [E01.370.350.700.725]

Radiography, Thoracic [E01.370.350.700.730]

Tomography, X-Ray [E01.370.350.700.810] +

Likewise "ICU" is not a MeSH term; instead, you would be directed to use "Intensive Care Units." As an alternative to using the above print sources, you also could find the appropriate MeSH terms on line via the MeSH Browser at the National Library of Medicine's (NLM) Medical Subject Headings Web Page.

Most of the new generation of software designed to search the Medline database (e.g., PubMed, OVID) now includes automatic term mapping. For example, in NLM PubMed's, unqualified terms that are entered in the query box are matched (in this order) against a MeSH (Medical Subject Headings) Translation Table, a Journals Translation Table, a Phrase List, and an Author Index.

What if the terms you originally generated do not have exact MeSH equivalents? In these cases, you also should be sure to include YOUR terms is a secondary search of Medline's title and abstract fields. In our example above, although 'ICU' and 'X-rays' are not a MeSH terms, they still may appear in either the title or abstract of a pertinent article. Searching for (and combining) YOUR terms in these locations will ensure that you do not miss articles with applicable content.

Other databases/indices use different controlled vocabularies. For example, were you planning to search the Cumulative Index to Nursing & Allied Health Literature (CINAHL), you would use its subject heading tree or permuted index (similar to MeSH, with about 70% overlap in terms). Were Excerpta Medica (EMbase) to be included in your search plans, you would access its terms index, called EMTREE (available in both print format and online). On the other hand, were you doing educational research, the starting point would be the Thesaurus of ERIC Descriptors, published by the National Library of Education's Educational Resources Information Center. Last, if your topical area were to include psychology, you would access PsycInfo's Thesaurus of Psychological Index Terms.

Below is a general summary of the key steps to follow when mapping your terms to a database's controlled vocabulary:

1. After dissecting your topic/question, map all your terms individually to the controlled vocabulary terms used in that database. If you get back "0" citation, check your spelling. In large database indices, most ever word in the English language appears.
2. Be sure to explore the hierarchy (outline) of terms used in the database (in Medline, this is called the MeSH 'tree'). This gives you an essential perspective on broader, narrower and related database terms. Include these as applicable.
3. If your terms do not exactly match those in the database's controlled vocabulary, include them in a separate search of the title and abstract fields.
4. Use phase searching (e.g., 'health students') sparingly. In general, uses of phrases that do not map to database terms results in a VERY narrow perspective, since the search engine will find only articles with that exact phrase in the title.

Step 4 - Combining Key Words/Descriptors (Boolean Logic)

After individually mapping out all your terms, it is time to begin combining them. You combine terms using the Boolean operators OR, AND and NOT. For a good tutorial on the use of Boolean logic in searching databases, click here.

Boolean OR. Use the Boolean OR operator to expand your search to citations indexed any of two or more key terms. This is the best place to start when you have multiple database terms with similar meanings. Using our prior example, by searching on 'critical care' OR 'intensive care' OR 'intensive care units' you would immediately find all those articles that are indexed on ANY of these 3 MeSH terms (a much wider set than either alone).

Boolean AND. Use the Boolean AND operator to narrow your search to citations indexed simultaneously under two or more key terms. Referring to our earlier question on whether daily chest x-rays improve outcomes in intensive care patients, searching on 'radiography, thoracic' alone and 'intensive care units' alone would yield thousands of references for each. However, by searching on 'radiography, thoracic' AND 'intensive care units' you would immediately find only those articles that are indexed on BOTH key terms (a much narrower subset of each).

Boolean NOT. Use the Boolean NOT operator to exclude from your search results any citations indexed under the specified keyword or term. The effect is to further narrow your search results. For example, by adding 'NOT (child OR infant)' to our search strategy on X-rays in ICU, we would purposefully exclude articles about children or infants.

Step 5 - Modifying Your Search Strategy (Narrowing, Expanding, Limiting)

You need to modify your search when you get back either too much or too little information. A special problem for searches in fields or areas subject to rapid change is the time lag between publication of a journal article and its indexing in a citation database.

Narrowing Your Search

You know you are getting too much information when the number of article retrieved is huge (thousands) and many of the titles or abstracts indicate content not directly related to your specific area of inquiry. When you retrieve too much information, you must narrow your search.

Most citation databases provide several ways to narrow your search. These include (1) combining terms with the Boolean operators AND or NOT (covered above), (2) using more specific terms, (3) using subheadings, (4) focusing on major topics only, (5) applying methodologic filters or (4) setting limits to your search.

Use More Specific Terms. As a simple example of using more specific terms, refer back to the MeSH tree on Diagnostic Imaging. Assume your specific topical interest is interventional radiography and pleural disease. However, you initially use 'radiography' as your key word. The result will be retrieval of articles covering any or all of the MeSH terms under Radiography. The narrower search on 'Radiography, Interventional' would result in fewer and more pertinent titles.

Use Subheadings (Carefully!). Many citation databases provide subheadings as a way to narrow your search. For example, the MeSH system used by the National Library of Medicine includes nearly a hundred useful subheading that can be applied to MeSH terms (click here for a table of representative subheadings and their abbreviated forms). As an example, suppose you were interested specifically in the safety/hazards of newer versus older mammography units. Searching on the key word 'mammography' in Medline would yield thousands of unrelated references to articles covering the broad spectrum of topics related to mammography. On the other hand were you to add selected subheading to you search -- perhaps instrumentation (IS), radiation effects (RE) and complications (CO), you would immediately narrow your search down to the most pertinent citations. To include subheading in your search, you typically add their abbreviated form to your search, e.g., 'mammography/IS.' Because they are less reliably than subject headings, you must be very careful using MeSH subheadings. Limiting a search using subheading should only be done as a last resort, and only if combining terms with the appropriate Boolean operators and/or using more specific terms does not sufficiently narrow your search.

Focus on Major Topics Only. A special MeSH subheading, 'Major Headings (MX)' can also be useful in narrowing your search. This is because Medline assigns key words as being either a major or minor pointer to a given citation. A simple keyword search on a MeSH term yields all citations indexed under that keyword (major or minor). When an article is indexed by a 'minor' term, that term is only peripherally related to the article contents. For example, any article mentioning chest x-rays would probably rate an inclusion of 'Radiography, Thoracic' as a minor term. But only those whose primary focus is on thoracic radiography would fall under the 'Radiography, Thoracic/MX' designation. Using the OVID interface to search Medline, you can narrow you search to articles in which your subject heading is a major term by selecting the 'Focus' box for that term.

Apply Methodologic Filters. Given the size and inclusiveness of the larger databases like Medline, even good use of content-oriented keywords and operators will in many cases not be sufficient to 'net' the best evidence. For this reason, many experts recommend the use of methodologic filters, i.e., keywords or terms that narrow the focus to the specific type of study or level of evidence that you are looking for. The National Library of Medicine provides a PubMed Clinical Queries page which you can use to automatically apply methodologic filters that will narrow your search to articles primarily focused on therapy, diagnosis, etiology or prognosis. The Duke University Medical Center Library provides an excellent description and listing of methodologic filters for use when search Medline via OVID:

http://www.mclibrary.duke.edu/subject/ebm/ebmfilters.html

Limiting Your Search. After you combine your terms, you may want or need to limit your search results. Common limits include year of publication (to eliminate old or obsolete works), language (to retrieve only English publications), and type of publication (original research, reviews, letters, editorials, etc). Medline also include keywords useful in limiting your search by animal vs human subjects research and by age group (infant, preschool child, child, adolescent, middle age, aged). Limiting should always be done for a legitimate, defined reason, not just to decrease the number of article retrieved.

Expanding Your Search

You know you are getting too little information when you the number of article retrieved is small (usually ten or less). When you retrieve too little information, you must expand or widenyour search. To widen your search you can (1) combine terms with the Boolean OR (covered above), (2) use more general terms or (3) use truncation.

To use more general terms to search a database, you need to access that database's hierarchy of terms and move up to a higher outline level. If using Medline, you would access the MeSH tree and simply 'climb up' to a bigger branch. If, for example, 'mammography' yields too few citations when searching Medline, you might try 'radiography' or even 'diagnostic imaging.' As an alternative, some database search engines (e.g., OVID; EMBASE) include an EXPLODE function that will search for and retrieve citations using your selected term and all of its more specific or related terms.

Truncation is used to search the root of a word and retrieve all variations which begin with that root. In Medline, you truncate the root using an asterisk. for instance 'bacter*' will find all terms that begin with the letters bacter, e.g., bacteria, bacterium, bacteriophage, etc. In general, you should not use truncation when mapping your terms. Also, avoid using truncation when phrase searching. In most databases, phrases that include a space in a word after the asterisk will NOT be included. For example, in Medline, infection* includes infections, but not infection control.

Step 6 - Selecting, Retrieving and Storing Your Source Documents

Selecting Source Documents

In the past, the source document selection and retrieval process always followed searching, i.e., you would write down or print out the full citations resulting from a search, and then 'go to the stacks' to find hardcopies of the titles that appeared most promising. There a quick review of the abstract or text would determine its relevance. Then it would be 'off the the Xerox machine' with the relevant selections in tow (and lots of change).

Electronic databases and full-text access has changes this process substantially. Now much of the selection process occurs concurrently with the search. With most electronic databases now providing ready access to abstracts in addition to article citations, you can use this information as a preliminary screen to determine the relevance of the source material. Unfortunately, not all citations have abstracts and some abstracts provide insufficient information on which to base a selection decision. Fortunately, with the growth of the Internet and sophisticated electronic library and databases services, a substantial proportion of source articles now can be dynamically linked to during the search process. Obviously, the ability to review the full article while searching makes the selection process much more efficient.

Of course, not all source material is available as full-text online. A thorough search will always necessitate accessing traditional 'hardcopy' resources, meaning either a trip to the library (with 'lots of change' for Xeroxing) or a (for distance learners) a request copies via the UMDNJ retrieval service (see below).

Retrieving and Storing Source Documents

Once you have selected your source materials, you need to obtain and store the requisite information in a format that provides you with ready access as you organize and write your literature review.

Online Full-Text. Depending on the journal, access to full-text may be open to all, limited by (recent) volume or issue, or by subscription only. When searching OVID via the UMDNJ gateway, you can link to the full-text of selected journal articles directly via OVID and/or via the Findit@UMDNJ service. The following figure shows how this occurs within an OVID search of Medline (on Adult Respiratory Distress Syndrome). For article 2, a full-text version of the article is available via the OVID service (the 'Ovid Full Text' link). Clicking on this link will typically open a new window with the full-text article as a Web page (HTML format). This Web page full-text can be saved (using your Browser's File -> Save commands) but often will NOT include relevant tables or figures. If available, ALWAYS select the Adobe Acrobat (PDF) version of the article (listed in the OVID Link, if available). The PDF format retains the full 'look and feel' of the published journal article, but requires that you have the Adobe Acrobat reader installed on your system. If after viewing the PDF full-text you decide to 'keep' it, use the PDF tool bar save icon to save a copy to your computer's hard drive, being sure to give the file a name that make sense. generally this filename should include key bibliographic information. For example, I used the following filename for article 2: heyland_ccm_33_1549_2005.pdf.

Articles 3 and 4 can be accessed via linking to the Findit@UMDNJ service. Clicking on this link for article 3 brings up the following window, indicating that there are three potential sources for its full-text version accessible through the UMDNJ Libraries:

As with OVID full-text, it generally is best to find and use the source that provides a PDF version of the article you are considering for selection and to save it as described above.

If you already have a citation and want to know if you can access it as full-text, the best place to start is at the UMDNJ Libraries Full-Text Resources page. Select Electronic Journals, then Article Finder and follow the instructions provided.

Traditional Hardcopy. Not all source material is available electronically as full-text. For this reason it is essential that you will retrieve the full citations, abstracts and locations of all documents you consider potentially applicable to your review. For primary source materials (i.e., journal articles), the location normally is provided with the citation (author, journal, year, volume, issue, page). For secondary sources (books, review annuals, etc), you will need the specific library call number.

To obtain the actual journal articles, you must go directly to the journals in which they appear. Research libraries are the major sources of journal collections. Once you locate essential articles, you should photocopy them to provide for more convenient access. Photocopies of articles in journals not held by your library usually can be obtained quickly via interlibrary networks. For remote learners, the UMDNJ George F. Smith Library provide a retrieval service, accessible at:

http://www.umdnj.edu/librweb/newarklib/forms/illform2.html

Organizing Your Source Documents

Depending on the scope of your review, you can choose to organize your source documents manually or electronically. Typically, a good manual system involves creating a 'mini' catalog of alphabetized and cross-referenced index cards that provide the full citation for each document, a summary abstract, pertinent keywords and any notes or comments you wish to maintain.

Given the time involved in setting up a large manual card index, you best bet is to organize your sources using the appropriate computer software. Although you can use a generic database program like Microsoft Access to organize your sources, there are several excellent personal bibliographic software programs available that are designed specifically for this purpose. Typically, these personal bibliographic software programs combine the utility of a database (search, store and retrieve capabilities) with the added capacity to (1) search online databases and library catalogs; (2) automatically download citations; and (3) create bibliographies in selected style formats. Obviously, anyone serious about bibliographic inquiry should consider using such software.

Currently, there are four major commercial bibliographic software programs available. Click on the product name for more information or to download a demo versions (this listing does not constitute either an endorsement or recommendation to purchase):

• EndNote
• Reference Manager
• Procite
• Biblioscape

EndNote is now available as a free download (PC or Mac) for those with UMDNJ Library accounts/EZProxy access. To download your free licensed copy of Endnote go to:

http://www.umdnj.edu/librweb/access/EndNote.html

Step 7 - Abstracting and Evaluating/Critiquing Your Source Documents

Abstracting. If an article or document has no abstract, you should prepare your own. Personal abstracts should at least summarize the method and findings in a way that is most germane to your own needs and interests. Although index card were once used for this purpose, it is more common these day to include your abstracts in a personal computerized bibliographic database. These personal abstracts always should be headed by the full reference exactly as it will appear in your final reference list. The abstract itself normally is divided into the following three sections: (1) background (purpose and hypotheses or research questions), (2) method or procedures, (3) findings and (4) conclusions. You will probably not use all of this information in writing your review; but since you do not know what and how much you will need, it is wise to have it all (particularly if you borrow the source materials and have to return them).

Evaluating. If your review includes original research, you must also critique each research article according to some standards of quality. Such standards typically focus heavily on the research design and methods, with which a good reviewer must be familiar. Evidence-based standards go a step further by focusing on the type of research you are critiquing (i.e., therapy, diagnosis, prognosis or etiology/harm). For a good overview of evidence-based critical appraisal with separate criteria for these types of research see:

http://umdnj.edu/idsweb/idst6400/crtical_appraisal.dochttp://denison.uchsc.edu/SG/main.html#critical_appraisal

IDST6400 (Evidence-Based Literature Review) provides the following four critical appraisal worksheets to assist you in evaluating original research studies:

• Critical Appraisal Worksheet - Therapy
• Critical Appraisal Worksheet - Diagnosis
• Critical Appraisal Worksheet - Harm/Etiology
• Critical Appraisal Worksheet - Prognosis

For more detail on the use and evaluation of the medical literature, see the full text versions of the JAMA Users' Guide to the Medical Literature available online at:

http://umdnj.edu/idsweb/idst6400/crtical_appraisal.doc

Step 8 - Organizing and Writing Your Review

Once all source documents have been reviewed, selected, abstracted evaluated, it is time to organize and write your review. A good reviewer avoids the common tendency simple to summarize each article in turn. Instead, the reviewer should initially identify major research trends related to the topic under study and provide justification (ideally both theoretical and practical) as to its importance. Major studies with impact on problem should be reviewed and their validity evaluated, with conclusive or controversial findings highlighted. Key findings should be integrated and synthesized and inferences drawn regarding the implications of the findings. Last, the reviewer should clearly identify any unresolved questions, so that others can use the review as a springboard to future research on that topic. For the current criteria used in the UMDNJ-SHRP MSHS and PhD in Health Sciences Programs to evaluate students' literature reviews, click here.

Below is a basic outline helpful in organizing a research-based literature review:

1. Identify Your Topic/Issue/Problem Area

a. Narrow focus to problem area

b. Provide clear statement of the problem or issue

c. Delineate research trends related to problem/issue

d. Provide justification (theoretical and practical need)

2. Describe the Methods Used to Find and Select Chosen Literature

a. Clearly identify your data sources (Medline, CINAHL, etc)

b. Define your search strategy(ies)

c. Describe your study selection criteria (inclusion/exclusion)

d. Delineate the number of studies screened versus those included (ideally using a table format)

3. Review the Major Studies with Impact on the Problem or Issue

a. Provide a meaningful organizing framework

b. Evaluate the validity of each study's finding

c. Highlight conclusive/controversial or conflicting findings

4. Integrate and Synthesize Your Findings

a. Draw inferences regarding implications of findings

b. Identify any unresolved questions

In terms of writing a review, you should first secure and read several existing review articles published in scholarly journals representative of your field. This will give you a feel as to the appropriate structure and style accepted by your peers. Be sure to strictly follow the recommended style format (e.g., ICMJE/AMA, APA, etc). Last, be sure to follow elementary rules of usage and basic principles of good composition.

For additional guidance on writing literature reviews, see the following:

• Literature review paper (Univ of Wisconsin - LaCrosse)
• Literature Reviews (Univ North Carolina Writing Center)
• Writing literature reviews (Claremont Graduate University)
• Bem, D.J. (1995). Writing a review article for Psychological Bulletin. Psych Bull, 118, 2, 172-177.(oriented for a psychology audience, but excellent nonetheless)
• Muskal, F. Guidelines for the literature review paper (Univ of the Pacific School of Education) (oriented for an education audience, also includes excellent general pointers)

For some good examples of literature reviews written by prior students in the UMDNJ-SHRP MSHS and PhD in Health Sciences Programs, click here.