The following resources offer a critical look at traditional measurements of impact:
In the past, peer reviewed journals have been the standard for measuring the quality and importance of scholarly literature. However, the growing number of journals and articles being published has placed significant strain on this process. Part of the problem is attributable to a rate of publication that is far outpacing the pool of available referees. With scant incentive available for referees in the form of credit in the tenure and promotion process, this shortage is expected to continue.
Despite its strong track record in filtering research, the peer review system has come to show signs of weakness. The authors of the altmetrics manifesto point out that “peer review…is slow, encourages conventionality, and fails to hold reviewers accountable…[and] given that most papers are eventually published somewhere, peer-review fails to limit the volume of research.” This prognosis is not good and is unsustainable over the long term. In response, new alternative forms of peer review have begun to develop that take a more open and transparent approach to the traditional peer review model:
Open Peer Review: Open peer review begins with an author or editor posting an unpublished manuscript online in a comment-enabled web environment, and then inviting peers to contribute comments and criticisms. The argument is that open peer review not only allows for transparency in the peer review process, but also enables a wider variety of input, including cross-discipline critique and more technical, "non-scholarly" input.
Post-Publication Peer Review: