Citation and content analysis are two methodological techniques used by criminologists for a variety of purposes. Citation analysis is a way of evaluating the scholarly impact of a scholar, scholarly work, journal, book, or academic department within a discipline. Content analysis allows criminologists to systematically examine the contents of a book, article, television program, or other work. It is often used as a way of discovering patterns within individual works or bodies of work. Both are quantitative methods that are less likely to be affected by personal bias than other techniques.
II. Citation Analysis
A. What Is Citation Analysis?
B. Alternatives to Citation Analysis
C. Advantages of Citation Analysis
D. Problems With Citation Analysis
E. Other Sources of Citation Analysis Data
F. Concluding Remarks
III. Content Analysis
A. What Is Content Analysis?
B. Types of Content Analysis
C. Advantages of Content Analysis
D. Disadvantages of Content Analysis
E. Concluding Remarks
IV. Conclusion and Bibliography
Citation and content analysis are two methodological techniques used by criminologists for a variety of purposes. Citation analysis is a way of evaluating the scholarly impact of a scholar, scholarly work, journal, book, or academic department within a discipline. Content analysis allows criminologists to systematically examine the contents of a book, article, television program, or other work. It is often used as a way of discovering patterns within individual works or bodies of work. Both are quantitative methods that are less likely to be affected by personal bias than other techniques. This research paper discusses each technique in detail, including a review of the advantages and problems of each method.
II. Citation Analysis
A. What Is Citation Analysis?
Citation analysis is a technique that is widely used to evaluate the impact and prestige of individual scholars, academic journals, and university departments within a discipline. The technique may also be used to determine the impact an individual scholarly work (a book or journal article) has on subsequently published research in the field. In addition to its application in criminology, citation analysis has been used in disciplines such as medicine, economics, physics, sociology, and psychology. The rationale for using citation counts as a measure of research eminence was best explained by Rushton (1984): “If psychologist A’s work has been cited 50 times in the world’s literature that year, and psychologist B’s only 5,A’s work is assumed to have had more impact than B’s, thereby making A the more eminent” (p. 33).
Citation analysis first came to prominence in criminology in Wolfgang, Figlio, and Thornberry’s 1978 book Evaluating Criminology. The authors used the technique to determine the most-cited American books and journal articles in criminology between 1945 and 1972. Twenty years later, their research inspired Cohn, Farrington, and Wright to return to the topic in their 1998 book, Evaluating Criminology and Criminal Justice. In this book, they used citation analysis to examine the most-cited scholars and works in a variety of American and international journals in criminology and criminal justice over a 10-year period.
These books, and many other studies that use citation analysis in criminology and criminal justice, have resulted in a considerable amount of controversy over the acceptability of citation analysis as a scientific technique. Although many scholars find the results to be an interesting and important contribution to the field, others find this research threatening and may even actively oppose the publication of articles using citation analysis. However, recent research does suggest that the approach is both valid and reliable and is an important tool for measuring prestige and influence in criminology.
Citation analysis provides researchers with an objective and quantitative method for determining the impact on the field of a scholar, journal, work, or department. It assumes that if a specific article or book is frequently cited, many scholars find that work important and valuable. In addition, citation analysis assumes that citations reflect the influence of a given work on the field, so that if two researchers were working independently on the same problem, they would both cite the same material. Although there is some question as to whether citation counts accurately measure the quality of a highly cited work, they are used to measure that work’s influence or prestige.
Currently, there are two main methods commonly used for gathering citation data. The first method is to use citation indexes, especially those produced by the Institute for Scientific Information. These include the Science Citation Index, the Social Sciences Citation Index (SSCI), and the Arts and Humanities Citation Index. These indexes list literally millions of bibliographic references made in thousands of journals published throughout the world. For the purposes of criminological research, the SSCI clearly is the most useful of the three.
The second method of gathering citation data involves examining reference lists of journals, scholarly books, textbooks, and other works in a given field and counting the number of citations made of a given scholar, scholarly work, or journal. Although this method is considerably more labor-intensive and time-consuming, it does permit researchers to avoid a number of problems that are inherent in the use of SSCI and other citation indices (discussed later). This technique was pioneered by Cohn and Farrington (1990) and has since been used successfully, both by them and by other researchers in criminology.
B. Alternatives to Citation Analysis
Unlike citation analysis, most other measures of scholarly prestige and influence do tend to be vulnerable to personal bias. One of the most commonly used methods is peer review. In this method, a researcher may survey directors or chairs of criminology departments, or survey members of a scholarly society, such as the American Society of Criminology or the Academy of Criminal Justice Sciences, and ask them to rank academic journals, books, or PhD programs in criminology and criminal justice. However, it is clear that the results of this type of survey may be affected by the respondents’ personal opinions. For example, scholars who have served as an editor or member of the editorial board of a particular journal may be more likely to give that journal a higher ranking because of their familiarity with it. Similarly, when ranking PhD programs, scholars may be inclined to give the program from which they graduated or where they currently are employed a higher ranking, or they may be inclined to give a rival department a lower ranking. In essence, regardless of the respondent’s desire to be objective, personal preferences or knowledge may creep in and affect his or her responses to this type of survey.
A related method is to consider which individuals receive scholarly prizes or are elected to major offices in scholarly societies. This method is similar to peer review, because in most cases, recipients are chosen or elected by members of the field. However, these methods all tend to identify the same individuals. They are also equally vulnerable to bias, because it is obviously easy to be influenced by personal likes or dislikes of the individual, department, or scholarly work under review.
Another method that is used to measure prestige and influence is to count the number of journal publications of an individual criminologist or of the entire faculty of a criminology department. This method clearly is far more quantitative and objective than that of peer review; however, it measures only productivity and does give a clear indication of influence. Just because an article is published in a journal does not mean that it will be read and/or cited or that other scholars will view the article as being important in any way. In addition, many of these studies attempt to weight the publications in some way, such as by the prestige of the journal. This often reduces the objectivity of the method, because journal prestige usually is determined by peer review. Finally, the rapidly increasing number of journals in the field of criminology is creating many additional outlets for publication and may serve to inflate publication rates, thus reducing the validity of this measure.
Overall, it appears that none of these methods provides as straightforward, objective, and quantitative measure of scholarly influence and prestige as citation analysis. Although citation analysis has its shortcomings, it appears to be more valid and reliable than any other method. “The overwhelming body of evidence clearly supports the use of citation analysis as a measure of scholarly eminence, influence, and prestige” (Cohn, Farrington, & Wright, 1998, p. 4).
C. Advantages of Citation Analysis
One of the most notable advantages of citation analysis is that, unlike other measures of scholarly influence or prestige, such as peer rankings, citation analysis is objective and quantitative and is not affected by any personal bias. Whether the data are obtained from a citation index such as SSCI or from the reference lists of journals and other scholarly publications, they are readily and publicly available and cannot be affected by any personal bias, even that of the researcher.
The question of reliability and validity of citation counts has been examined by a variety of researchers. Research in a wide variety of academic disciplines supports the relationship between citation counts and other measures of scholarly influence, intellectual reputation, professional prestige, and scientific quality. Citation counts have been found to be highly correlated with scholarly productivity, peer ratings of professional eminence, scholarly recognition (e.g., election to the National Academies of Science), and the receipt of scholarly prizes (e.g., the Nobel Prize in physics). There also appears to be a strong correlation between citation counts and ratings of the prestige of university departments and doctoral programs. Researchers also have found citation counts to be correlated with peer rankings and journal publications. Rushton (1984) stated, “It is fair to say that citation measures meet all the psychometric criteria for reliability,” and concluded that “citation counts are highly valid indices of ‘quality’” (p. 34)
Another concern frequently raised by those who oppose the use of this method is that it focuses on quantity rather than quality of citations. However, this appears to be an untenable position, given that citation counts are highly correlated with other measures of prestige and influence. Although it has been suggested that a high citation count may indicate a past contribution to the field, rather than a current or ongoing one, research suggests that, in general, scholars tend to cite more recent works rather than older ones. Researchers such as Cohn et al. (1998) have suggested that the influence of scholarly works tend to decay over time as they are supplanted by more recent work. One recent study estimated that social science research works have a half-life for citations of only about 6 years (Cohn & Farrington, 2008).
D. Problems With Citation Analysis
1. General Problems
Citation analysis is not a perfect measure of scholarly influence; the technique has a number of problems. One objection to citation analysis is that it may be biased against scholars who work in a narrow or less-populated specialty. If there are only a few researchers working in a particular area, there will be fewer articles published in which those researchers can and will be cited. As a result, these scholars, regardless of how influential they are in their area of expertise, are less likely to be frequently cited.
Another concern is that citation analysis counts all citations to a given work equally and does not distinguish among positive, negative, and neutral citations. In other words, just because a work is highly cited does not automatically mean that it is looked upon favorably by others in the field. However, a number of researchers have considered this and have found that the vast majority of citations appear to be positive or neutral. Very few citations appear to be critical or negative.
Several researchers have suggested that “recipe” articles, such as those outlining a new personality test or explaining a statistical technique, tend to be highly cited. However, in general this does not appear to be the case. Recent research into the most-cited scholars and works in criminology and criminal justice journals (Cohn & Farrington, 2007a, 2007b, 2008) has not found methodological works or their authors to be among the most cited.
Citation counts also may be affected by a scholar’s productivity. In general, the most-cited criminologists tend to be older and more established in the field. They also tend to be fairly prolific writers and to have long publication records. However, Cohn and Farrington (2007a, 2007b) have found that a scholar’s high ranking may be either a function of the large number of different works cited (versatility) or a function of a large number of citations of one or two major works (specialization). Of course, it is possible for a scholar to be both versatile and specialized (e.g., to have a large number of works cited, with one or two of these receiving most of the citations).
Another concern is that an author’s choice of which articles and/or authors to cite may be influenced by social factors, such as personal likes and dislikes, attempts to please journal editors, a desire to inflate individual or departmental citation counts, or even a preference for citing same-sex authors. For example, a scholar may cite others in the same department to boost overall departmental citation counts or may avoid citing faculty from a rival department to reduce their citation counts. Although it is difficult to constantly cite oneself (self-citation) as a way of inflating one’s own citation count without being rather obvious about it, other more devious methods, such “You cite me, I’ll cite you,” could be used. However, for such an approach to work, the scholars would still have to have a reasonably high publication rate in good quality journals.
Chapman (1989) discussed the issue of “obliteration by incorporation”:
Certain scholars may be so eminent and prolific in their fields that, although their names appear in the body of an article, they can elude the ordinary counting process because the writer neglects to list them in the references at the end of the text. In psychology, textual mentions of Freud, for example, often are not referenced and are thus underrepresented in citation counts. (p. 341)
It has been suggested that a similar situation may occur in criminology with mentions of scholars such as Karl Marx.
However, despite these difficulties, citation analysis is generally accepted as a valid technique and is increasingly becoming more popular and more widely used in the sciences and social sciences.
2. Problems With Citation Indexes
The SSCI lists all bibliographic references made in an extremely large number of social science journals. The index is an extremely useful tool for general bibliometric research but has several problems when used for citation analysis.
First, the SSCI does not include references in all published works. According to the SSCI Web site, as of 2008, the SSCI fully indexes over 2,000 journals in 50 social science disciplines, as well as individually selected items from over 3,300 leading scientific and technical journals. However, not all criminology journals are included. For example, neither Criminology and Public Policy nor Violence and Victims currently are included in the SSCI.
In addition, although the SSCI includes citations of books and book chapters, works cited in books or book chapters are not indexed. It is possible that this may lead to a bias, because books appear to be highly significant in criminology. Cohn and Farrington (2007a, 2007b, 2008) have examined the most-cited works of the most-cited authors and found that most were books, rather than journal articles.
Another problem with the SSCI is that it lists only the initials and surnames of cited authors, making it almost impossible to distinguish between different people with the same last name and first initial. For example, Cohn and Farrington repeatedly have pointed out the difficulty of determining which of the citations of “J. Cohen” belong to Jacqueline Cohen and which refer to other individuals, such as Jacob Cohen or Joseph Cohen. Similarly, it is difficult, if not impossible, to distinguish between the various R. Berks, P. Brantinghams, and D. Smiths. This confusion is only increased if citations include or omit an author’s middle initial (e.g., making it more difficult to distinguish between E. G. Cohn and E. S. Cohn) or use a “nickname” first initial (such as R-for-Robert vs. B-for-Bobby Brame). In addition, citations of married female scholars may appear under more than one surname (e.g., Ilene Nagel, Ilene Bernstein, Ilene Nagel-Bernstein). Other scholars who hyphenate or change their surnames create similar difficulties for the citation analyst attempting to use the SSCI.
The SSCI also creates a bias against junior authors in collaborative works, because it lists citations only under the name of the first author. Therefore, an individual who is not listed as the first author of scholarly work will not be found in the SSCI. This may penalize younger scholars, scholars whose names come later in the alphabet (for articles where authors are listed alphabetically), and wives (e.g., in research by Sheldon and Eleanor Glueck, Sheldon’s name is almost invariably listed first).
Citation indexes such as the SSCI also may contain errors. First, because it assumes that all citations in the source journals are accurate, any clerical or other errors in the original reference lists, such as misspelled names, incorrect dates of publication, or incorrect initials, are transferred to the SSCI. There is considerable evidence to suggest that this assumption is, for the SSCI at the least, questionable. For example, Cohn and Farrington (1996) pointed out that “Farrington” is often misspelled as “Farringdon.” Similarly, “Hirschi” is frequently misspelled as “Hirsch,” “Hirsh,” or “Hirshi.” In addition, Hirschi’s citations have been found under “P. Hirschi” and “L. Hirschi” as well as the correct “T. Hirschi.” When dates of articles are incorrect in the original reference list, they are entered into the SSCI under the incorrect year. In general, studies of citation accuracy have found that between 25% and 30% of all citations contained errors. Of course, this does not include any errors that may be made by the staff at Institute for Scientific Information who compile the SSCI.
Finally, the SSCI includes self-citations. Self-citations occur when a scholar cites his or her own work. It is, of course, perfectly reasonable and justifiable for scholars to cite themselves, especially when their work is building on their prior research. However, self-citations do not indicate the influence of the cited work on other scholars, and a study of citations as a measure of influence on others in the field should omit self-citations. Because only the first author of a cited work is included in the SSCI, it is extremely difficult to exclude self-citations when using the SSCI to obtain citation counts.
E. Other Sources of Citation Analysis Data
An alternative to employing citation indexes such as the SSCI is to examine reference lists of journals, textbooks, scholarly books, and other sources within a field and count the number of citations to a given scholar, work, or journal. This technique has been used successfully by Cohn and Farrington to produce a sizable body of citation analysis research in criminology. Although it is more time-consuming, this method is arguably more accurate and avoids many of the problems that are inherent in the use of the SSCI. For example, it allows the researcher to examine all authors of a cited work, not just the first author, and to exclude all self-citations. It also provides a knowledgeable researcher with an opportunity to correct at least some of the errors found in citation lists. For example, if the researcher sees a reference to a “T. Hirsch,” he or she will be able to determine whether this citation is of a work by Travis Hirschi or that of a different author. This method also allows the researcher to attempt to distinguish between multiple authors with the same surname and first initial, because many journals list first names (not just initials) in their reference lists, or even scholars with the same full name (e.g., the Australian vs. the American Patrick O’Malley or the Australian vs. the British David Brown).
The technique developed by Cohn and Farrington is fairly straightforward, although admittedly somewhat tedious and time-consuming to carry out. For each journal that is used as a data source, they either download the reference pages for every article from an online full-text source into a word processing program or enter the pages into a computer from a printed copy of the journal using an optical scanner. For references with multiple authors, duplicate listings are made of the reference, with each coauthor listed first. Mistakes in reference lists are corrected when found. When all references for a given journal are entered into a computer file, they are sorted into alphabetical order, and this list is examined to determine the number of times each name appears. When known, citations of scholars with multiple names (e.g., Ilene Nagel/Bernstein) are amalgamated. If reference lists did not include first names or middle initials, Cohn and Farrington would, when necessary, check references against original publications to distinguish between, for example, the various D. Smiths (Douglas A., David D., David E., David J., etc.), the various J. Cohens (Jacqueline, Jacob, Joseph, etc.), and the various different scholars with the same name (David Brown, Richard Sparks, Richard Wright, Patrick O’Malley, etc.). In some journals, references occasionally specify “et al.” rather than listing all authors. In these cases, the original works were checked, when possible, to obtain the names of all coauthors and insert them into the data file. To maximize the accuracy of the data, this method requires an extensive and detailed knowledge of criminology authors and publications.
F. Concluding Remarks
Overall, citation analysis is an extremely quantitative, objective method of exploring trends in scholarly impact and prestige. The increasing availability of electronic journals and books throughout the world is helping to make this technique more accessible and may increase its use in the near future. Researchers may in the future use citation analysis to examine how changes in theoretical, empirical, and political issues affect research and scholarly influence. They may also use this method to determine patterns and trends in research topics and perhaps predict changes in key issues over time.
III. Content Analysis
A. What Is Content Analysis?
Content analysis, or textual analysis, as it is sometimes known, allows scholars to systematically study and classify the contents of an individual work or body of work, often to determine the presence of certain words or concepts. It is defined as “the identifying, quantifying, and analyzing of specific words, phrases, concepts, or other observable semantic data in a text or body of texts with the aim of uncovering some underlying thematic or rhetorical pattern running through these texts” (Huckin, 2003, p. 14).
Content analysis is applied not only to scholarly works (e.g., journals or books) but also to both the print and visual media (e.g., newspapers, television, movies). Other sources of data for content analysis include book chapters, interviews, conversations, speeches, historical documents, and so on. Content analysis may be applied almost anywhere communication occurs.
Content analysis has been used in criminology to study a wide variety of topics, ranging from how newspapers discuss community policing (Mastrofski & Ritti, 1999) to how television newscasts cover crime (Chermak, 1994). Wolfgang et al. (1978) used content as well as citation analysis when examining the field of criminology. They read, classified, coded, and rated more than 4,400 works, ranking them on scientific merit. More recently, Richard Wright (1995) conducted a number of content analyses of criminology and criminal justice textbooks, looking at topics such as the coverage of women and crime, the coverage of career criminals, treatment-of-choice theory, and the coverage of deterrence.
Content analysis allows researchers to study not only how messages are conveyed but also what meaning those messages convey. Like citation analysis, it is a primarily quantitative method of analyzing data. However, one of the key elements of content analysis is coding the concepts to be studied, and this may involve some subjectivity. For example, if a researcher wants to examine all references to prisons, a wide variety of terms may be relevant, including not only prisons but also jails, supermax, corrections, and so on. Which terms are used will depend on the individual researcher and the research question.
B. Types of Content Analysis
There are two main types, or categories, of content analysis: (1) conceptual and (2) relational. These were described in detail by Busch et al. (2005).
1. Conceptual Analysis
Conceptual analysis is what most people think of when the term content analysis appears. It is occasionally referred to as thematic analysis. In essence, it involves selecting a concept to be studied and determining how often that concept appears in the material being examined. For example, conceptual analysis could be used to determine how often terms relating to youth gangs are mentioned on a local newscast, appear in a local newspaper, or are mentioned in speeches by key government officials (or candidates).
To increase objectivity, particularly if multiple individuals are involved in the research, it is necessary for the terms in question to be identified in advance, so as to ensure intercoder reliability (in other words, to make certain that all researchers focus on the same specific words or word patterns). For example, in a study of youth gangs, other terms might also be used to refer to the concept in the text being examined; the researchers must decide in advance which terms imply “youth gangs” so that they will know which to terms include when and if they appear (e.g., synonyms such as juvenile gangs or teen gangs, as well as the names of specific gangs, such as “Crips” or “Bloods”). This step also is necessary when there is only one researcher examining the data, to ensure consistency throughout the data coding process.
It is also necessary for the researchers to decide whether to study presence or frequency. When the researcher is looking only at the presence (or absence) of a concept, it does not matter how often the relevant term appears in the text; the researcher cares only whether the term appears at least once. Therefore, a newspaper article that mentions youth gangs only once would be considered equal to one that focuses on the topic and mentions youth gangs repeatedly throughout the article. In a frequency study, on the other hand, a key term will be counted each time it appears. Measuring frequency rather than simple presence allows the researcher to assign a level of importance to the term. For example, one might conclude that a political candidate who mentions youth gangs 25 times during the course of a campaign speech considers the problem to be more serious than a rival candidate who mentions youth gangs only once during a speech.
Content analysis in criminology has also focused on manifest content, looking at the amount of coverage, usually measured by column print inches or pages, given to specific topics or individuals. Manifest content analysis researchers in criminology, criminal justice, and deviance have used length-of-coverage measurements in a variety of different ways. For example, Cohn et al. (1998) used content analysis to examine the amount of coverage given to scholars in introductory criminology textbooks and identify the most influential scholars on the basis of page coverage. Other researchers have used the number of inches of print in newspaper columns as a way to measure the amount of publicity devoted to various news stories.
2. Relational Analysis
Relational analysis, which is also known as semantic analysis or concept mapping, examines the relationship among various concepts within a given text. Relational analysis not only identifies concepts within the text but also explores the relationships between the various concepts. For example, examining the terms that appear next to or near the phrase community policing in newspaper articles may give insight into community attitudes toward or views about community policing. The basic idea behind relational analysis is that the individual concepts have no inherent meaning in and of themselves but they instead gain meaning as a result of their relationship with other concepts in the text. Researchers generally look at three main aspects of the relationship among the concepts being studied: (a) the strength of a relationship shows how strongly the concepts are related, (b) the sign indicates whether concepts are positively or negatively related, and (c) the direction refers to the type of relationship (e.g., does one concept precede another?).
C. Advantages of Content Analysis
There are a number of advantages to content analysis. First, it is both quantitative, or objective, and qualitative, or impressionistic. Much of the process is clearly objective; counting the prevalence or frequency of occurrence of set of words or phrases in a television news program, for example, is strictly quantitative and does not consider any underlying semantic features that may be found from an examination of the context of those phrases. However, the technique also allows for more subjective operations, such as allowing the individual researcher to determine whether specific television programs are pro- or antiviolence. Simply counting the number of violence-related terms that appear in the programs may not make it clear whether the programs support or oppose violence. However, an examination of the context in which the terms are used, although more subjective and somewhat less reliable, may provide more insight into this research question.
Another advantage of content analysis is that it is an unobtrusive research method. Experiments and surveys frequently require researchers to interact with research participants during the data collection process in an abnormal or unnatural way. This interaction may affect the responses of the participants, who may omit key pieces of information, either deliberately or inadvertently. Texts and other types of media, on the other hand, are not affected by being read and analyzed by the researcher. For example, if a researcher is studying politicians to determine their views on the problem of youth gangs, the researcher could interview each politician individually. However, it is possible that the politicians might respond to the interviewer’s questions with answers that they believe the interviewer wants or that make the politician look better. On the other hand, an examination of each politician’s speeches would provide a more unbiased record, because the transcripts of the speeches cannot be altered to appear more favorable. Therefore, content analysis allows the researcher to reduce bias during the data collection process.
An additional strength of content analysis, in particular conceptual analysis, is that it generally is highly replicable. In other words, a different researcher, using the same coding system, should be able to produce the same results.
Content analysis also is extremely flexible and convenient for researchers. There are no surveys to conduct, no experimental subjects to test, no focus groups to conduct. The researcher can perform the analysis on his or her own schedule, rather than having to coordinate with research participants. In addition, because this methodology does not involve human participants, there are fewer ethical issues to take into consideration.
D. Disadvantages of Content Analysis
There are also several problems with content analysis as a research method. First, it is often very time-consuming and labor-intensive. The process is not as simple and straightforward as it may appear at first glance. Defining the categories, for example, may be a difficult process when multiple researchers are involved, and pretesting is essential to ensure that nothing is overlooked and that there is no confusion of terminology. In addition, the process requires a considerable amount of time. A study of prime time television programs to study violent content does require the researcher to watch the programs; even with commercials deleted, this will require a large time investment.
Another criticism of content analysis is that it frequently focuses on only the surface issues. In an effort to be more quantitative, the researcher may simply conduct word counts without looking at the context in which those words appear. In other words, the researcher focuses on the individual words, rather than on their meaning, by ignoring the contextual aspects of the communication.
Some scholars have pointed out that when content analysis is more qualitative, such as when one is conducting various types of relational analysis, the coding used frequently becomes more subjective and open to interpretation. This has the effect of increasing error and reducing reliability. To deal with this problem, multiple coders may be used, and interrater reliability measures may be applied.
E. Concluding Remarks
Overall, content analysis provides criminologists with opportunities to study, examine, and make inferences from a variety of print and other media. It allows researchers to expand their horizons and explore new concepts and new relationships among those concepts.
Both content and citation analysis have a variety of problems; however, these techniques also offer scholars many advantages that are not as readily available with other methods. They provide an objective method for studying both texts and the scholars who produce them. Both use existing documents, or other forms of communication, as sources of data for analysis, rather than involving human participants in research. Both have the potential to be cumulative; that is, as further documents become available for study, they may be incorporated into the research, allowing researchers the opportunity to study trends over time. Both methods are widely used in criminology, and both are somewhat controversial. However, although there is some controversy within the field as to their use, both approaches appear to be both reliable and valid, and both clearly have widespread uses within the field of criminology and criminal justice.
- Busch, C., De Maret, P. S., Flynn, T., Kellum, R., Le, S., Meyers, B., et al. (2005). Overview: Content analysis. Retrieved March 22, 2008, from http://writing.colostate.edu/guides/guide.cfm?guideid=61
- Chapman, A. J. (1989). Assessing research: Citation-count shortcomings. The Psychologist, 8, 336–344.
- Chermak, S. M. (1994). Body count news: How crime is presented in the news media. Justice Quarterly, 11, 561–582.
- Cohn, E. G., & Farrington, D. P. (1990). Differences between British and American criminology: An analysis of citations. British Journal of Criminology, 30, 467–482.
- Cohn, E. G., & Farrington, D. P. (1996). “Crime and justice” and the criminology and criminal justice literature. In N.Morris & M. Tonry (Eds.), Crime and justice: A review of research (Vol. 20, pp. 265–300). Chicago: University of Chicago Press.
- Cohn, E. G., & Farrington, D. P. (1998a). Assessing the quality of American doctoral program faculty in criminology and criminal justice, 1991–1995. Journal of Criminal Justice Education, 9, 187–210.
- Cohn, E. G., & Farrington, D. P. (1998b). Changes in the mostcited scholars in major American criminology and criminal justice journals between 1986–1990 and 1991–1995. Journal of Criminal Justice, 26, 99–116.
- Cohn, E. G., & Farrington, D. P. (2007a). Changes in scholarly influence in major American criminology and criminal justice journals between 1986 and 2000. Journal of Criminal Justice Education, 8, 6–34.
- Cohn, E. G., & Farrington, D. P. (2007b). Changes in scholarly influence in major international criminology journals between 1986 and 2000. Australian and New Zealand Journal of Criminology, 40, 335–360.
- Cohn, E. G., & Farrington, D. P. (2008). Scholarly influence in criminology and criminal justice journals in 1990–2000. Journal of Criminal Justice, 36, 11–21.
- Cohn, E. G., Farrington, D. P., & Wright, R. A. (1998). Evaluating criminology and criminal justice. Westport, CT: Greenwood Press.
- Gordon, R. A., & Vicari, P. J. (1992). Eminence in social psychology: A comparison of textbook citation, Social Sciences Citation Index, and research productivity rankings. Personality and Social Psychology Bulletin, 18, 26–38.
- Huckin, T. (2003). Content analysis: What texts talk about. In C. Bazemore & P. A. Prior (Eds.), What writing does and how it does it (pp. 13–33). Mahwah, NJ: Lawrence Erlbaum.
- Mastrofski, S. D., & Ritti, R. R. (1999). Patterns of community policing: A view from newspapers in the United States (COPS Working Paper No. 2). Washington, DC: U.S. Department of Justice, Office of Community Oriented Policing Services.
- Regoli, R. M., Poole, E. D., & Miracle, A. W., Jr. (1982). Assessing the prestige of journals in criminal justice: A research note. Journal of Criminal Justice, 10, 57–67.
- Rushton, J. P. (1984). Evaluating research eminence in psychology: The construct validity of citation counts. Bulletin of the British Psychological Society, 37, 33–36.
- Shichor, D. (1982). An analysis of citations in introductory criminology textbooks: A research note. Journal of Criminal Justice, 10, 231–237.
- Steiner, B., & Schwartz, J. (2006). The scholarly productivity of institutions and their faculty in leading criminology and criminal justice journals. Journal of Criminal Justice, 34, 393–400.
- Thomas, C. W. (1987). The utility of citation-based quality assessments. Journal of Criminal Justice, 15, 165–171.
- Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage.
- Wolfgang, M. A., Figlio, R. M., & Thornberry, T. P. (1978). Evaluating criminology. New York: Elsevier.
- Wright, R. A. (1995). The most-cited scholars in criminology: A comparison of textbooks and journals. Journal of Criminal Justice, 23, 303–311.
- Wright, R. A. (1996). Do introductory criminology textbooks cite the most influential criminologists? American Journal of Criminal Justice, 20, 225–236.
- Wright, R., & Cohn, E. G. (1996). The most-cited scholars in criminal justice textbooks. Journal of Criminal Justice, 24, 459–467.