Introduction
When thinking about citation analysis and impact factors, you should try to consider the perspective from your own field of study. Some possible questions include:
- How do you measure success of an article or book in your discipline?
- How do you measure the success of a scholar in your discipline?
- Is there discussion of citation analysis/impact factor particular to your discipline?
- Are impact factors used in promotion and tenure decisions regarding professors in your department? In evaluating job applicants?
Purpose of Citations
Attribution of ideas/research
- for confirming/illustrating a point
- for disputing/correcting/questioning
- for use of methods, tools, design, definition or data
Provide proof that position is well-researched by providing
- holistic view of research
- literature review
- primary and historical sources
Help disseminate useful knowledge
- demonstrating other points of view
- supplementary information
Give formal credit for research as a normative research practice
- for purpose of association and community
- to meet funding, legal, and governmental requirements
What is Citation Analysis
Citation Analysis is the examination of the frequency and patterns of citations in articles and books. It is not a new phenomenon but has been used in the sciences since the 1960s and the social sciences and humanities since the 1970s. Citation analysis includes but is not limited to citation counts, statistical analysis (e.g. h-factor), and data visualization.
How it can be used
Evaluate the impact of individual research
- Impact is neither good nor bad but merely a measure of how aware people are of given publications through citation
- Must not be conflated with quality of research
- A popular example of this is Google PageRank which has used website links as a measure of the impact of a given web page
- Can be susceptible to self- or endogenous-citation practices. Example Another Example
Discovery of related research
- Publications are typically cited within the same narrow field of research
- This factor can be extremely limiting depending on the nature of the citation analysis tool you use
Mapping impact in a field of research or journal
- Can be used to rank or visualize the impact of a publication, scholar or journal
- Citation Analysis is commonly used in the sciences as one measure in tenure application packages
- Possible source for discovering referees for a field of research
Deciding on a publication
- Can be a factor on choosing a journal to publish in
- Notably agnostic of fields of research and may not favour interdisciplinary work
Metrics
There are a number of ways that scholarly publication can be measured and these are called metrics. Scholarly metrics depend upon traditional avenues of publications but as scholarly communication extends on to the internet alternative metrics (altmetrics) are being devised to evaluate scholarly work.
Scholarly Metrics
H-index from a plot of decreasing citations for numbered papers
Journal Impact Factor (used by ISI Web of Science)
A Journal Impact Factor (JIF) is measured by dividing the number of current citations a journal has in a given year, by the number of articles published in the two previous years. The JIF is used to indicate the relative importance of a journal within a given field and a higher JIF is seen as providing more “authority” or “weight” to a researchers’ work.
Eigenfactor
A non-commercial alternative to ISI Web of Science's JIF. The Eigenfactor algorithm also takes into account the significance of the citations coming into a journal so that citations from important journals are weighted more highly.
Acceptance Rate
Typically expressed in percent, acceptance rate is compares the number of articles selected for publication compared to the number of articles submitted. For example, the very prestgious journal Nature publishes fewer than 10% of the articles submitted. In general, a low acceptance rate is associated with high prestige. See The Journal of Universal Rejection for a humorous response to acceptance rate analysis.
Individual author/article citation counts
Citation Tracking (CT) counts the number of times an article has been cited by other scholars. The common belief is the higher the number of cited references, the greater the impact the research has within the field. Sources for CT include: Web of Science, Google Scholar, PLoS, BioMed Centra, plus databases on EBSCO and ProQuest.
H-index
A statistical means of calculating author influence. An h-index is the number where the number of articles published by an author intersects on a graph with the number of citations for each article. For instance, an author with h-index of 10 has published 10 papers that have been cited at least 10 times each. The h-index is the first (and most well-known) of many author metrics.
The use of the Journal Impact Factor as "the primary parameter with which to compare the scientific output of individuals and institutions" has been criticized by a number of scholars and organizations, most notably here.
Alternative Metrics
Download counts and View Counts
The number of times an article has been downloaded and the number of times that it has been viewed. View counts may not represent actual interest in obtaining an article. Download counts can be used to demonstrate use of articles for purposes other than citation such as education, background research, presentation, or use in grey literature such as white papers or reports.
Authority scores (e.g. Klout)
Authority scores attempt to measure how many people you reach through your own contacts, their amplified reach (e.g. how many people are followed by people who follow you on Twitter) and how often your content elicits a response (i.e. a comment or reply).
Tools
Many publishers (e.g. Wiley) and scholarly databases (e.g. PsycINFO) now support searching for cited references. This feature is generally limited to articles within their own database. The following academic and altmetric tools are commonly used for more thorough citation analysis.
Academic tools
ISI Web of Science
- One of the most standard commercial tools for finding Journal Impact Factors, Citation Counts and other author metrics
- Favours sciences over social sciences and humanities
- Journal-based (i.e. Uses journal articles as a source for citations. Books, patents, etc. are included only if they have been cited by a journal article.)
- Video example on conducting Author Citation Reports
- Tutorial for using ISI Journal Citation Reports
InCites
Google Scholar Citations; Sample Profile UBC Math Professor Nassif Ghoussoub
- Video Tutorial of Google Scholar Citations
- Allows authors to curate their own list of publications
- Calculates citation counts, h-index, and i10-index
- Tracks conference proceedings, chapters in edited volumes, and books, but not consistently
- Better for humanities and many social sciences than ISI Web of Science or Scopus
Google Books
- Tracks references in books to books, book chapters (and journal articles)
- Finds unique citations, i.e. not included in Google Scholar,Web of Science or Scopus
- Books to journal citation ratio, roughly: Philosophy 4:1 |Sociology, Fine Arts 3:1 | Psychology 1.5:1 | Physics .001:1
Publish or Perish
Altmetrics
ImpactStory
- A new experimental tool co-authored by UBC postdoctorate researcher Heather Piwowar that combines internet metrics with traditional scholarly metrics.
ReaderMeter
- Another experimental tool that uses readership metrics derived from Mendeley, an online reference management application with social networking features.
Other Tools
Top Ten (10) indices measuring scholarly impact in the digital age from The Search Principle, April 2012
Readings
Fang, F. C., & Casadevall, A. (2011). Retracted science and the retraction index. Infection and Immunity, 79(10), 3855-3859. doi: 10.1128/IAI.05661-11
García-Pérez, M. A. (2010), Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h indices in Psychology. Journal of the American Society for Information Science and Technology, 61: 2070–2085. doi: 10.1002/asi.21372
Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science and Technology, 62(11), 2147-2164. doi:10.1002/asi.21608
McDonald, John (2003). The Book is Dead: Citation Rates in the Humanities and Social Sciences. In Strauch, Katina (ed.) Charleston Conference Proceedings (21st, Charleston, South Carolina, November 1-3, 2001) 143-152. [Google Books Link, not all pages viewable]
Moed, Henk F. (2005). Citation Analysis in Research Evaluation. Information Science and Knowledge Management 9. doi:10.1007/1-4020-3714-7
