Guidelines for the Literature Review Paper

by Fred Muskal, Ph.D.
School of Education, University of the Pacific

The expansion of empirical research in the past several years has increased the need for periodic assessments and reviews in all fields. Literature reviews (LRs) have, therefore, become an increasingly important form of scholarship. Generally, LRs evaluate some body of empirical research by organizing it according to some theory, model, ideal of logical coherence, or conception of wholeness. They may be used to answer questions about the state of knowledge in a field or to generate theory, for example. Their utility is shown by the number of publishers and journals specializing in this form of research.

The plan of this paper is to examine the literature review in a number of ways. The first part explores the conceptual structure of the LR, focusing primarily on principles of organization. The next section deals with some of the mechanics of developing an LR, while the third deals with some characteristics of the traditional chapter two. The final summary attempts to synthesize everything in a useful manner. Each section focuses on slightly different characteristics; taken together, they should help the student develop an LR of acceptable quality.

The purpose of an LR is to organize and evaluate some body of literature in a manner which yields some conceptual insight to the reader. There are several factors to consider here. First, it is important to define the area of review clearly and to achieve some completeness. The matter of definition is important. It may place impossible burdens on the reviewer (and reader) to examine methodology in educational research. It is more reasonable to look at qualitative methodologies in classroom observational studies. The point is clear enough: define the area of review carefully. The topic should be specific, clear, manageable, and complete.

Second, completeness in most LRs means covering all recent research while providing some historical background as introductory material. Generally, the author chooses historical material which fits his/her primary principle of organization. This permits the introduction of the topic in a way which orients the reader to the author's approach; the LR should evaluate in some significant way a body of literature.

This means that the author cannot simply list various pieces of research as a form of annotated bibliography. The LR is a piece of scholarship, not just a rehash of several related articles. It must organize a discussion or argument around certain organizational principles selected by the author. The LR is an assessment, an analysis which synthesizes an argument from some well-defined body of literature. This should be enough for a basic definition. The most important point in this brief discussion is that the author must develop some coherent principle(s) of organization from his/her analysis of the literature. It is to this point that we now turn.

Principles of Organization

The most crucial aspect of any LR is the manner in which it is organized. The author's obligation, after reading and understanding the literature well, is determining how to best inform the reader. This requires considerable attention to selecting categories, themes, and principles which form the body of literature. They need to be identified and organized properly to construct and adequate LR. For our purposes, there are two sets of principles which may be used to organize LRs. Conveniently enough, we can call these primary and secondary.

Secondary principles include organizing by chronology and topic, for example. These principles are secondary or subordinate because they do not inform or enhance the reader's understanding in a conceptual way. Instead, they are merely convenient ways to array information. A review of learning theories, for example, organized around both chronology and topic would b e no more than a lengthy annotated bibliography, a string of isolated short reviews.

Such an LR might organize learning theorists chronologically so that one would read about Thorndike before Gagné. Any comparisons drawn by the author likely would be limited by the organizing principles, e.g., Thorndike is an early learning theorist, while Gagné builds on his work today. Notice that the principle of organization tends to inhibit rather than expand the reader's understanding, similarly using topic only as an organizing principle might well lead to a mini-review of each theorist. Again, the series of mini-reviews would probably inhibit rather than facilitate deeper conceptual understanding.

The basic point here is that these principles, chronology and topic are reasonable and useful primarily in a subordinate way. They clearly are part of defining the area of review, but do little to facilitate deeper, conceptual understanding of literature. In short, they are but a partial form of organization which must be subordinated to stronger conceptual principles if the LR is to teach the reader. Some discussion of primary principles of organization and a continuation of the learning theory example should help clarify the matter.

Primary principles of organization are designed to yield to the reader conceptual insights and understandings. They must be developed from the author's analysis of the literature. Generally, they can be conceived of as the author's argument. For example, an LR might be organized around the argument that cognitive considerations dominate effective ones in learning theory. Such an LR would focus on the balance between cognitive and affective components in various learning theorists. Another conceptual approach to learning theory might examine the relationship between methodology and theory characteristics, statistically verified theories may be substantially different from clinically verified theories. Note that these examples are different kinds of understandings than are available from chronology or topic alone. Let us move to the real world for some better examples of how to organize LRs.

The following examples are drawn from various reviews dealing with education. Essentially, they are excerpts from the introduction to LRs which are considered quite good. Each is organized around some conceptual principle or argument which is made clear in the introduction. The first example is drawn from the work of Rebecca Barr and Robert Dreeben.

There are two traditions of research on school effects that have developed and remain in almost complete isolation from each other. First, there is a classroom instruction tradition consisting of a large and inconclusive literature on the impact of instruction upon learning. Long before anyone misinterpreted the Coleman Report of 1966 as showing that schools have no effect on achievement, there was ample evidence that the research on classrooms has little to say about the connection between what teachers do and what students learn. Second, a more recent tradition dating to the Coleman Report examines school effects using a quasi-production function formulation to examine the connections between school resources and achievement.

We will argue not only that the two traditions have a connection but that the agenda addressed by one is integral to the agenda addressed by the other. Whatever the inadequacies of past work the study of the impact of classroom instruction on learning (or achievement) is directly relevant to the application of production function models to schooling. In the production function literature the sources of within-school variation have either been ignored or handled improperly. Moreover, the most complete understanding of the macrosocial aspects of schools and school systems will surely contribute to better understanding of how classrooms work and students learn.1

This is the entire introduction. It gives the plan of the paper clearly and concisely, including the major argument and its general assessment of the literature. Notice that it tells the reader exactly what is going to happen in the paper and the main point of the LR.

A second example draws from the work of Walter Doyle. The excerpts from his introduction show how to use chronology as a subordinate or secondary principle in service of his primary principle of organization --an examination of how methodological themes shape inquiry in research on teacher effectiveness:

Teacher effectiveness research has occupied a conspicuous place within the spectrum of scientific inquiry in education. Interest in the question of what distinguishes superior from inferior teachers has flourished since the early 1920's and by mid-century had stimulated an impressive number of studies (Domas & Tiedeman, 1950). The quest was rejuvenated in the 1050's with the formation of the AERA Committee on Criteria of Teacher Effectiveness (American Educational Research of the Handbook of Research on Teaching (Gage, 1963a). This developmental path underscores the axial role of the teacher effectiveness question within the broader field of research on teaching. Even a cursory inspection of recent literature suggests that effectiveness inquiry continues to attract the resources of researchers, funding agencies, and professional organizations.

Throughout its history, research on teacher effectiveness has faced problems of productivity, methodology, and theory (for general discussions, see AERA, 1952, 1953; Barr, 1939, Berliner, 1976; Gage, 1972). Reviewers have concluded, with remarkable regularity, that few consistent relationships between teacher variables and effectiveness criteria can be established (Barr, 1961; Dunkin and Biddle, 1974; Getzels & Jackson, 1963; McKeachie & Kulik, 1975; Medley & Mitzel, 1959; Morsh & Wilder, 1954; Rosenshine, 1971l Rosenshine and Furst, 1973; Stephens, 1967). Although optimism is more apparent in recent writings (Flanders & Simon, 1969; Gage, 1977; Good, Biddle, & Brophy, 1975; Rosenshine, 1976a, 1976b), a general perception of low productivity would seem to prevail. The productivity issue is associated in part with several basic, largely unresolved, methodological problems that have impeded attempts to compare studies, integrate findings, or apply results to teacher education (Borich, 1977; Flanders, 1973; Glass, 1974; Heath & Nielson, 1974; Shavelson & Dempsey-Atwood, 1976). There has also been a continuing concern for the adequacy of theoretical foundations in the field (Dunkin & Biddle, 1974; Gage, 1963a, 1964; Guba & Getzels, 1955; Rabinowitz & Travers, 1953; Turner & Fattu, 1960). Researchers have shown considerable ingenuity in generating variables that might possibly relate to effectiveness indicators. There are, however, a few theoretical grounds for selecting variables or for interpreting available findings.

This chapter is based on the premise that many of the questions of productivity, methodology, and theory are related to the underlying assumptions which shape inquiry. The discussion can be characterized, therefore, as an analytical review of the conceptual foundations of research on teacher effectiveness. It is focused on alternate ways of thinking about teaching and their implications for asking questions and interpreting answers about effective teachers. Of necessity, methodological issues are considered only as they illustrate the structure or assumptions of teacher effectiveness research. No attempt has been made to compile an up-to-date list of current findings. The review emphasizes common features among investigations, an approach that does not do justice to the diversity of work in this field. Despite these limitations, there is reason to expect, on the basis of previous attempts to examine the form of the teacher effectiveness question (Gump, 1964; Olson, 1972; Stephens, 1967), that an analysis of conceptual issues is a fruitful direction for inquiry. 2

Another noteworthy aspect of Doyle's introduction is his review of reviews in the second paragraph. His discussion is well-organized and has no need to use quotes from any of the sources cited. Rather, he makes his points and offers support for them in the form of footnotes. In his 25 page LR, Doyle quotes nothing longer than a single sentence because his principle organization is not covered directly in any of the reviewed research. The only point here is that quotes are not necessary for a good LR, though of course they may be used to good effect.

To summarize a bit, an LR is constructed around a well-defined area of academic literature. It is an essay organized primarily by conceptual principles which enable the author to synthesize in a manner which deepens the reader's understanding of the material. With the LR's conceptual organization, secondary principles such as chronology or sub-topics may be useful ways to organize the presentation of material. To return to the learning theory example, within the primary organization of looking at clinically and statistically grounded theories it may be useful to array material chronologically. Thus the author would discuss Rousseau before Piaget and Thorndike before Gagné. Whatever principles of organization inform the LR necessarily derive from an intensive analysis of the literature. The simpler mechanics of analysis and writing loosely form the next section.

The Mechanics of Developing and Writing LRs

The mechanics of doing LRs range from tips on collecting information to important points in writing them. This section will cover a lot this material in a nuts and bolts fashion, hopefully providing a few usable ideas. The development of any LR begins with some system of note-taking.

Everybody knows how to take notes, but doing it productively is too often another matter. One common system is to use 5" x 8" cards. The size of the card is a matter of personal preference, but cards offer many advantages. First off, they can be shuffled endlessly as the author tries new ways of organizing and reorganizing material. Ultimately one uses a lot of space on the card, so leave the top open for various organizing schemes. Use the first or second line for a complete bibliographical entry--every fact of publication available. Do this for each entry, even if there are 25 readings from the same book. This saves a lot of work later on, especially unnecessary trips to the library. The rest of the card is used for ideas and quotes from the work. Usually, one starts with some organizing scheme in mind, so use some of the top space for a preliminary classification scheme such as type of research or type of theory, e.g., once notes have been collected on most of the material available, the work of outlining begins.

Outlining is an overlooked and underused tool. It is a terrific way to begin thinking on paper and developing ideas. Most of the themes which emerge from the literature are captured and analyzed best through outlining. Essentially, outlining is a more formal approach to shuffling note cards; it's the first attempt to develop a sound principle of organization for the LR. Use outlining to develop and check ideas. They are a shorthand technique for examining your own consistency. By the way, never throw away outlines or notes on outlines. There is always some idea that one NEEDS at some future point, and it usually is part of a discarded outline. Save everything. One stuffed file folder can save a lot of grief.

At some point, you will become tired of outlining and start work on the first draft . LRs, like most academic writing, need a number of drafts. The first draft generally helps by getting down on paper all those things you noted on your cards. If you have outlined carefully and developed some primary principle of organization, the first draft may be good. In any event, pay attention to your thinking as you write. Writing is a powerful learning process, and you may discover flaws in your outlining, that is, your organization.

Probably the best thing to do on a first draft simply is to finish it. Note your problem areas and mistakes, but get it all down on paper. A first draft gives some degree of control over your material because you have had to cover it three times by now: the initial reading, note taking, and the first draft. The first draft usually functions as a formal set of notes from which the final principles of organization are worked out after careful editing.

Editing is one of those things that always seems painful. Usually it is difficult to read your writing as if someone else wrote it, but that is the basic trick to editing. Divorce yourself from your writing and you will be able to look at it more clearly. One may prefer to use favorite crutches and make usual mistakes on the first draft and then eliminate them in editing. Editing usually is easier if you can put the work down for a day and come to it freshly and only as an editor.

Last, one of the keys to writing a good LR is developing a strong introduction. The examples in the foregoing section are both excellent. Note what they have in common. They show the author's control of the material by elaborating either the primary principle of organization (Doyle's) or the basic argument of an LR (Barr & Dreeben). An introduction ought to be a conceptual road map to the intellectual terrain to be covered in the LR. It is the author's way of preparing the reader to understand. In short, the introduction should give some indication of the scope and direction of what is to follow. As a summary of all this and some slightly new material, let us turn to the traditional dissertation chapter two.

The Chapter Two Literature Review

Most dissertations use a literature review as part of the overall study. While the form of this review may vary with different types of studies, basic purposes remain constant. In general, the literature review (LR) provides the reader with a major context for understanding both chapters 1 and 5 (in the traditional format). The LR should enhance the reader's understanding of the problem statement from Chapter 1 by reviewing research relevant to the formation of the problem, objectives, and/or hypotheses. It may review research relating to both independent and dependent variables, e.g., or elaborate some relevant theory. In a sense, it enables the reader to understand better why the author has chosen to perform this study. The LR's purpose for Chapter 5 is related, but takes a different form. (The student should understand that the LR, in its entirety, may or may not be appropriate for the content of Chapter 2 in the dissertation, one a prospectus has been approved.)

The LR must provide a context for locating the meaning of the study in Chapter 5. This means that the LR provides the general understanding which gives meaning to the discussion of findings, conclusions, and recommendations. This allows the author to demonstrate how his research is linked to prior efforts and how it extends our understanding of this general line of scholarly inquiry. The following typology of studies suggests some ways in which LR's may be organized by authors.

Experimental studies generally require the LR to focus on both independent and dependent variables and research relating to them. As the example of Doyle's introduction suggests, such LR may not contain a main argument. The relationship of independent to dependent variables offers a primary principle of organization, so long as research is reviewed and an annotated bibliography is avoided. This is a tricky main principle, but in general it suggests that independent variables be reviewed in a way which shows what they may cause and dependent variables be reviewed in a way which shows what causes them, i.e., as effects. This may entail reviewing findings in one section and methodological issues (conditions on the findings) in another, but it should give the reader a strong understanding of issues involved in the research.

Theory testing or theory generating studies add the requirement of elaborating some area of theoretical inquiry. This type of LR may raise more questions than it answers, but it should point out the state of our knowledge.

Historical studies usually don't have a separate LR, but incorporate its principles in the entire study. For such studies, it is probably more appropriate to have the student develop a background chapter which incorporates an LR.

Survey studies, depending on whether they attempt to be inferential, purely descriptive, or model building will require differing LRs. All LRs share common writing requirements, however.

Writing the LR

The LR must demonstrate the author's understanding of relevant research and how it is important to this study. Reviewing means drawing from the literature what is significant to the author's study. For example, stating that Freud's work assumes a biological basis to human behavior may be an important point for a study. Once the author makes this assertion, there is an obligation to support it with evidence. The evidence may take the form of a footnote or some discussion of Freud's relevant work. Note that the author need not and should not provide a complete description of each of Freud's works and what he attempted in them. Rather, the author need only point out to the reader how this assertion maybe supported. The LR is the work of the author, not a string of quotes.

Generally, the LR should be written in the author's own words. Some quotation is useful, but excessive quotation suggests borrowing from others rather than organizing one's own LR. Excessive quoting and description of discrete pieces of research really provide no more than an annotated bibliography for the reader. Annotated bibliographies usually are irrelevant to the purposes of the LR and mislead by leaving the reader to sort out what is really on the author's mind. This shifts the burden of explanation from the author to the reader and shows that the author has not accepted the responsibility of determining what is important to the study and why.

The clarity of the author's writing depends on a variety of factors besides careful organization. First, the subject-verb-object order of the English declarative sentence requires careful thought as long as the main idea is used as the subject of the sentence. Using the main idea as the sentence subject logically commits the author to declaring something about the topic. Only good can come of this. Uncertainty shows in writing when main ideas are buried in subordinate phrases or clauses. Such burying suggests that the author is not fully organized or does not have anything to say. Usually it is a sign of incomplete organization.

Another important factor in writing LRs involves identifying clearly the methodology underlying an author's statement. The word "finding" usually indicates the results of empirical research, while "argues" suggests conclusory material drawn from a number of approaches. Similarly, if an author does not draw from his work a conclusion you wish to make, use the word "suggests" to draw an inference. The point is simple, identify the rationale for material you are dealing with in your LR. Identify research findings, general conclusions, and suggested inferences as such. This is a matter of some importance because the author runs the risk of treating research findings with the same weight as arguments and opinions if no care is exercised.

The dissertation LR should not rely on bibliography alone to show sources. Near the beginning of a Chapter 2 LR, perhaps just after the introduction, the author should state clearly the extent of the literature search. What libraries were used? What data sources such as Datrix, ERIC, etc. were employed? This gives the reader some general sense of thoroughness. This may be significant if few sources exist on relevant topics.

Summary

This is a nuts and bolts summary of how to do an LR. It is an attempt to tie everything together in a sequence which is both logical and practical. To begin, pick a topic and define it carefully. Give careful consideration to your problem and its requirements. Do a quick outline to get some grasp of what you want and where you are going. Any topic is likely to be revised by the available research, so bear in mind that your first topic may be modified considerably by the time it emerges as an LR. That is why you need a flexible note taking system.

Start your reading by looking for some relevant literature review. If you're lucky enough to find one that seems pertinent, read it carefully, being sure to mark off what seems like the most useful bibliography. That should give you some beginning ideas about the utility of your topic. If it's too broad, narrow, shallow, etc., or if your conceptualization of the topic is not matched by the research, think it out a bit. Outline what you think is there and plan on what areas you may cover. For example, lay out the limits of the topic, methodological considerations, and potential research questions. Get some idea of what ball park you may be playing in and develop a preliminary taxonomy: studies of teacher effectiveness, studies of school effectiveness,

studies of teachers, studies of effective teaching methods, experimental studies, correlational studies, observational studies, etc. This gives you some preliminary way of organizing your material.

When new material doesn't fit your typology or taxonomy, go back to the outlining boards and start thinking again. This is simply a developmental process which should culminate in a well developed set of principles of organization. In short, develop a framework which covers the research in ways relevant to your purposes and you will have elaborated one or more primary principles of organization. Needless to say, you should also outline your organization to figure out if and/or how to integrate primary and secondary principles of organization. As you read, your typologies should become refined through further outlining. At this point you will have to decide if any of the research is extraneous to your evolving topic. If so, set it aside but don't throw it away. Another reorganization may make it central.

When you are convinced that there isn't much more to do, begin writing. Remember there's always more, but when you're convinced that more research will yield only diminishing returns and that you have some control over the topic you have most recently defined, then begin writing. Outline all those principles and organize the cards as well as possible (it's never that neat). You may have a different organizing scheme for each principle, so the cards may need precision shuffling for each section of your LR. Begin by writing the introduction. That should give some indication of how well you control your material and tip you off to any organizational principle problems. Then get the first draft done.

Write according to your organizational principles. Organize your cards to meet the demands of your outline, but let the flow of your argument and analysis be determined by your principles of organization. If something doesn't fit, it may not belong or the outline may be deficient. This may send you back to the drawing boards or raise large questions about how the second draft should be written. You may want to put something in where you originally planned, even if it doesn't belong there. Just write it out and make notes to yourself in the margin or parenthetically. The aim of the first draft always is to get it down on paper. Rewrite the introduction and conclusion. Does the LR fit the introduction? If not edit carefully. Remember it may be more useful to rewrite the introduction than the LR, so long as you preserve clear primary principles of organization.

Put the paper aside for a period of time and then reread it as if it were the first time. Evaluate it as a reader first, then begin to edit it. Are the primary principles of organization clear? Does the writing adhere to them. How can the main points be clarified? Is the organization sufficient or does it need reworking to make all your points clear? Work out an outline. Check on how much of the writing you can salvage. Some parts may be transferable to new sections largely intact. Others may need to be dropped. Most new writing will deal with organization and analysis; some often little, will deal with reviewing research.

The first draft is the slowest piece of writing in any LR. It takes time because the author's control over the material is not yet complete. First drafts usually have organizational and writing weaknesses deriving from the lack of control over the material. In effect, the first draft is the step which gives more complete control over the material to the author. Subsequent drafts and minor revisions invariably go faster. Writing speed accelerates as control increases. The second draft should be rather refined. Organizational principles should be clear, command of the literature apparent in the introduction; analysis should illuminate conceptual relationships and the whole LR should enhance conceptual understanding of material as well as substantive understanding.

To put it by the numbers, first do some thinking and outlining. (These two terms are synonymous because each implies the other.) Second, collect some data, i.e., read and take notes. Third, outline some preliminary organization scheme and primary principles. Fourth, write the first draft. Fifth, edit the first draft for weak argumentation, writing, etc. Sixth, get back to outlining and organizing the second draft. Repeat the process as often as necessary, but not any more than that.