Office Hours Recap: Peer Review for Open Textbooks

In this month’s Rebus Community & Open Textbook Network Office Hours we were joined by special guests to discuss peer review for open textbooks.

Anthony Palmiotto, Editorial Director at OpenStax, discussed the OpenStax approach to peer review, the first part of which follows a traditional in-development review model, similar to that of the major commercial publishers. During this process, they work to ensure a mix of different types of reviewers, profiles and representation of different regions, especially for disciplines where geographic and political balance is important. The second part of their review process involves a post-development feedback mechanism where anyone can submit feedback on a book. This feedback is then reviewed by faculty, with minor revisions being made monthly, and major revisions yearly. OpenStax aim to be as transparent as possible with these processes, listing reviewers at the beginning of each book, which also helps to establish credibility.

Karen Lauritsen of the Open Textbook Network shared the OTN review process, in which faculty who attend an OTN workshop are asked to complete a review of a textbook of their choice in the Open Textbook Library. Karen indicated that this process reassures faculty of the quality of available open textbooks, and encourages adoptions. She also noted that because the Library brings together texts from a wide range of sources, they have all undergone differing levels and methods of pre-production review. This highlights the current variability of open textbook review processes. Later in the session it was suggested that there might be an opportunity to standardise this in such a way that is useful for everyone working on open textbooks, similar to the Creative Commons licenses (i.e. providing a shorthand indicator of the type/level of review done to a book).

Deb Quentel from CALI (The Center for Computer Assisted Legal Instruction) shared her approach to finding reviewers, which is to call on authors for suggestions first, then on CALI’s editorial board of about 60 faculty and librarians, all with different areas of expertise. Deb gave some great advice for finding ‘strangers’ to bring in as reviewers, looking online for those writing and teaching in related areas. Once found, reviewers are given some guideline questions and two weeks to produce a 1-2 page memo, and are offered a small honorarium in compensation. Deb also mentioned that CALI often asks subject matter experts to proofread chapters as well, adding a second layer of review that can catch other errors or make additional suggestions.

Jon Udell joined us from Hypothesis and gave a short demonstration of how the current Hypothesis annotation tools can be used to review content. He also indicated a strong interest in engaging with the Community and finding test cases for Hypothesis’ current and future tools, so that those using them can be involved with the development.

Hugh McGuire of the Rebus Community capped off the session by framing the Rebus approach to the question of how best to manage peer review for open textbooks. He reiterated the Rebus objective to develop community-driven processes, and leveraging collaboration as the key to making open textbooks successful at scale. He discussed the different kinds of review we might expect to see develop, from a formal/traditional peer review, to an open review later in the publication process, to ongoing feedback from users of the book to the author. Last, he closed with an invitation to all who are interested in the topic to help us figure out what this should all look like by joining a working group.

The discussion continued on once the speakers wrapped up, touching on compensation, building community around books, the importance of transparent processes and the possibility of creating standardised measures or levels of peer review, similar to the Creative Commons license, where there’s a widely recognised ‘badge’ as a shorthand. The working group will pursue these questions and ideas and we encourage you to join us!

You can see the full video of the session here:

One comment

Leave a Reply