PeerRevYOU: A Tool for Media Students

Presenting to you: PeerRevYOU, a peer review platform for Media students. Thanks to this tool, students are able to review each other’s work before they hand it in to their teacher. We argue that this process could improve grades, that it will stimulate students’ critical thinking skills and that they will learn to give constructive criticism.

Reviewing: an Academic Debate
This project does not intervene in a specific new media debate, but rather offers a solution using new media. The contemporary debate about peer review is broad and mainly focused on the world of scientific research.

On the one hand, some authors, especially in the field of science and medicine journals, have raised some questions on the effectiveness of peer review. For example, David Shatz argues, that peer reviewers quite often fail to spot blatant errors (47). Furthermore, in the case of open/not-anonymous reviews, the work of well known researchers are more likely to be accepted, even though their work might not be of outstanding quality (54-57). Another issue he points out concerns reviewer bias. A reviewer, will be more positive about research results that fit his prior expectations (42-43).

On the other hand, several studies have underlined various benefits regarding peer review within educational environments. Topping claims that “even simple quantitative” feedbacks between students “can have positive formative effects in terms of improved scores” (267).  Nicol, Thomson and Breslin demonstrate that students, who receive feedback from peers, obtain a set of abilities which are not necessarily part of their educational program (120). While Mulder et al. argue that university students who participate in a peer review evaluation “gain a deeper understanding of the subject material and the requirements of assessment task” (671). They also think that pupils involved in this process can improve their academic results.

Needless to say, these studies, rather than considering peer review as a teacher’s feedbacks replacement, recommend to integrate feedback between students into the usual educational path.

From Draft to Craft: How does PeerRevYOU work?
Users upload their assignment to PeerRevYOU before a set deadline. Next, the user should download someone else’s (predetermined) assignment, and produce a review for this assignment (again before a set deadline). When both sides are done reviewing, and have uploaded their review again, users can shortly rate the quality of the feedback they’ve received. The distribution of work happens on a double-blind (both assignment and review will be anonymous) and one-to-one (in the set time, a user only reviews one assignment and in return receives one review) basis. We recognize that anonymity might encourage users to rush their review, and therefore produce sloppy work. To counteract this problem, we have implemented the aforementioned rating system. This feature not only lets users build up a reputation, but it will ensure that they are partnered up with other partners with a corresponding reputation. Users with a lower rank will be encouraged to try harder in order to receive better reviews in the future.

By logging into the site with a UvAnetID, the system imports the schedule from rooster.uva.nl. The student participates in the platform on a voluntary basis. The algorithm will try to match two users with each other, trying to match firstly same-seminar, then same-course, then same-year students. Only when a group has an uneven amount of students, it will create one looping system of three besides the duo’s. The complete workings of the platform (and reasons to participate) can be found in the FAQ or on our Tumblr page.

This slideshow requires JavaScript.

Figure 1: Mockup of PeerRevYOU (made using wix.com), showing the workflow of the platform.

The system is different from other platforms in multiple ways. Since it’s voluntary, only you decide to participate, without any teacher interference or prior setup needed. Since it’s possible to use the platform for smaller, weekly assignments, the threshold remains low. In many peer review platforms, the peer review is the final stage in the assignment. In PeerRevYOU, you can still change the assignment after receiving feedback: this means you get feedback before you’re graded!

Pitfalls of peer review
While designing this project, we ran into difficulties that forced us to be creative and think of solutions. For instance, a critical condition for the platform to work is that we need active users putting effort into reviewing. To address this issue, as mentioned before, we kept the threshold fairly low by offering a free service for small assignments as well as larger texts, and the service is to be used voluntarily. Besides that, we hope to welcome more users by encouraging teachers to implement the platform into their course schedule.

Essential for the development of skills is that the users provide good feedback. We tried to guarantee a certain level of feedback by incorporating a reward system: get a review after uploading one, climb in rankings etc. Secondly, users sign up for PeerRevYOU voluntarily and thus they are likely to be willing to put some effort into the process in order to improve their assignments and skills. Simply put: they know what they signed up for – the two-way-street of peer review.

Finally, we addressed the issue of plagiarism. Although the process runs anonymously, the system stores details on users, uploads and so on. Abuse or plagiarism can be reported and investigated by admins that have access to the stored data. Plus, because the UvA still uses Ephorus, here plagiarism will be detected altogether.

A Flexible Platform – a Variety of Options
In conclusion, we’ve created a conceptual peer reviewing platform for students of Media Studies at the University of Amsterdam. The platform employs a one-to-one, double-blind system, where reviewees can rate their reviewers after they have collected their feedback. We are convinced that, in time, this will improve the overall quality of reviews, which in turn will have its effect on the quality of work. In the limited amount of time available sometimes choices had to be made to reach progression in the process.

PeerRevYOU can grow to be a dynamic and fluid project, that reacts to the needs of its user base. If users seem to participate enthusiastically, it might be possible to implement receiving multiple reviews. Also, the platform might incorporate users outside the Media Studies department, and thus reach students of different disciplines. Having students review each other will slightly alleviate the amount of time teachers spend on giving feedback to students and provide students with extra feedback at the same time. If they choose to implement PeerRevYou in a given course, it might be useful to connect the platform with a plagiarism detection tool.

And that’s why PeerRevYOU is the peerfect tool for you!

References

Mulder, Raoul et al. “How does student peer review influence perceptions, engagement and academic outcomes? A case study.” Assessment & Evaluation in Higher Education, 39.6 (2014): 657-77.

Nicol, David, Avril Thomson and Caroline Breslin “Rethinking feedback practices in higher education: a peer review perspective.” Assessment & Evaluation in Higher Education, 39.1 (2014): 102-22.

PeerRevYOU. Selma Dorrestein, Giovanni Carta, Cristiaan van Wijk, Lisanne Buijze. 2015. <http://peerrevyou.tumblr.com/>.  

Shatz, David. Peer Review: A Critical Inquiry. Rowman & Littlefield, 2004.

Topping, Keith. “Peer Assessment between Students in Colleges and Universities.” Review of Educational Research, 68.3 (1998): 249-76.