Peer review is critical for the acceptance of open textbooks by the wider world. Perceptions of uncertain quality of open textbooks is often cited as a barrier to adoption. Good, transparent review processes will help solve this problem.
While many open textbooks are comprehensively reviewed, there is currently no accepted standard or process for review.
We’ve put together a Peer Review Working Group to help the Open Textbook community come together to develop a clear approach to review. (Would you like to join us? Go here!)
We’ve currently got around 20 members in this group, from as many institutions, including: the Open Textbook Network, OpenStax, SUNY, Oregon State, and Ryerson University. You can find a recap of our first meeting (including the full video) here.
During this meeting, we identified a number of challenges for reviewing Open Textbooks, and our group hopes to provide some solutions, with subgroups focusing on the following issues.
Please join us if you think you can help us figure out smart approaches to reviewing Open Textbooks.
Group 1: Open Textbook Review Board The Rebus Community has proposed to create a review board that can been leveraged to help find reviewers for open textbooks. Members of the board will be contacted when a book becomes available for review and make efforts within and possibly beyond their institutions to find volunteers. This group will define and help to launch the board. Join Group 1 here.
Group 2: Defining Categories of Review This group will work to define the purpose and structure of the different kinds of review that could be valuable to an open textbook. They will also help guide the development of dedicated tools and resources for each kind of review. Join Group 2 here.
Group 3: Environmental Scan of Peer Review Tools, Processes & Research This group will conduct a scan of the existing peer review market in order to inform the development of new review processes. Join Group 3 here.
Group 4: Creating Standard Markers for OT Review This group will discuss creating a standardised system to clearly identify the kinds of review that any given textbook has undergone. This will link to the work of group 2, who are defining different categories of review. Join Group 4 here.
Group 5: Recognition for Reviewers This group will discuss ideas of how to recognise and reward reviewer contributions. This may eventually feed into the creation of something similar to the Mozilla Contributorship Badges project, recognising all kinds of participation in open textbook projects, including peer review. Join Group 5 here.
Signup for the groups will stay open, but we encourage you to join by May 20 so we can start moving forward on each of them.
In this month’s Office Hours, organized by Rebus Community and the Open Textbook Network, special guests and participants discussed their experiences with OER workflows.
Watch the video or read a recap of the session below.
Guests included Allison Brown, SUNY Geneseo; Rebel Cummings-Sauls, Kansas State University; Billy Meinke, University of Hawaii; and Anthony Palmiotto, OpenStax.
Karen Lauritsen of the Open Textbook Network introduced this month’s guests, asking them to briefly describe the processes they use when making open textbooks.
Billy Meinke, an OER technologist at UHawaii, works with faculty at his institution to shepherd them through the process of creating OERs. He shared an OER workflow diagramthathecreated, which provides a high-level view of the major steps involved in adopting and adapting an OER: priming, pre-production, design, development, and publishing. In order to make for more efficient workflows, Billy said, OER training at all UHawaii campuses also now includes Pressbooks training, to make individuals comfortable with the software they use for book production.
Rebel Cummings-Sauls is the director for the Centre for Advancement of Digital Scholarship at Kansas State University and specializes in copyright and Open Access. KSU provides grant funding for faculty to create open access textbooks and resources. Rebel said KSU prefers faculty to complete the resources commissioned within a year, though projects can take longer. Payment is one incentive she uses to keep projects on track. Rebel mentioned that one factor that can cause delays is that faculty members often want more rounds of private feedback on their textbook before it goes public.
Anthony Palmiotto is the editorial director at OpenStax, which creates open textbooks that are competitive with market-leading texts for specific college courses and makes them available free on the OpenStax website, generally under CC BY licenses. The OpenStax workflow begins with preparation: collecting market research, competitive benchmarking, educational research, length requirements, and so on. OpenStax involves faculty in this process, reaching out to them via surveys and at conferences. Often those who get involved in this way will continue to work on the project in later stages. OpenStax selects a team of faculty to work on the book as authors and reviewers. Later stages of production include revisions, originality checking, art rendering, fact checking and accessibility checks, and XML production. Anthony said OpenStax books typically take 18 months to 2 years to reach completion.
Allison Brown joined us from OpenSUNY, and described the production process for SUNY’s open textbooks. After a manuscript has been received, the workflow includes peer review, author revisions, copy editing, typesetting, proofreading, and finally publication. She said pain points included transitions (when a manuscript moves between collaborators such as writers and editors; or across platforms, such as from Word to Pressbooks) and copy editing. She said it’s important to forewarn authors of the expectations of them post-copy editing. To make the workflow more manageable, OpenSUNY conducts a thorough needs assessment of each manuscript prior to production. Allison says one common thread that ensures successful book projects is healthy communication with authors. She is transparent with authors, staff and freelancers, and clearly outlines expectations at various stages of the production process. She also mentioned the importance of someone acting as project manager to ensure projects stay on track.
The ensuing discussion touched upon a number of topics. Among these, the group discussed how to involve students in beta use and testing of books post-publication. Rebel said her institution obtains feedback on the books through several methods–evaluations, surveys, quizzes and forms from students whose faculty use the text in their class–and that they receive the bulk of student feedback and input in the first semester of a book’s use. Participants asked how OpenStax operationalizes originality checks and incorporates ancillary resources. Anthony said they use iThenticate and conduct spot checks of sections of copy with Google. Participants asked whether proofreading, copy editing, and design is assigned to freelancers or done in-house. Allison and Rebel both said that it varies with project, with Allison adding that she sometimes outsources cover design to students. One of the major themes that came out of this discussion was the importance of educating faculty and students about CC licenses and their implications, as well as copyright and fair use guidelines. New Prairie Press’ Permission to Publish was shared as a resource.
Peer review is an integral component of traditional academic publishing, but in the world of open textbooks, there is currently no common standard for reviewing in open textbooks. To address this need, Rebus Community has formed a working group to explore peer review for open textbooks. The group held its first meeting May 3 to discuss how the community of open textbook practitioners could build a coherent strategy and best practices around peer review for open textbooks.
Below is a recap of the meeting. You can read next steps and how to get involved here.
Deb Quentel from CALI started the session by describing CALI’s editorial review process for the lessons and textbooks they produce. Deb said CALI has an editorial review board consisting of potential reviewers in specific subject areas that she reaches out to first when looking for reviewers. If no one has the relevant expertise, she asks board members to leverage their networks to find appropriate reviewers. If that fails, the author may suggest potential peers as reviewers. Finally, Deb scouts reviewers online from relevant websites and publications.
In CALI’s peer review process, reviewers are asked to look at subject matter accuracy and whether the content covers the terrain to be expected. Reviewers also evaluate the sophistication of the end-of-chapter questions that are included for students. Reviewers provide a no-more-than-two-page memo of feedback. At CALI, peer reviewers are anonymous and usually review one chapter of a text. About 75% of the chapters for any given book get reviewed. Deb outlines her process in further detail here.
Next, Hugh McGuire of Rebus Community outlined various types of pre- and post-publication review that he hypothesized may need to be done to an open textbook:
Subject matter expert review (a formal, pre-publication review, similar to traditional peer review)
Open review, in which anyone can make comments (less formal)
Student and faculty feedback on a published, beta version of the textbook
Ongoing feedback (how does feedback get back to the authors following publication?)
Revision and updates (especially for subjects that change frequently)
Karen Lauritsen from the Open Textbook Network gave an overview of OTN’s review process. All new OTN member organizations are able to send faculty to workshops as part of their professional development in open textbooks. In return, they are required to provide a light review for one book in the Open Textbook Library and are given a rubric from which to do so. Karen said that roughly 40% of these reviewers end up adopting a textbook from the library for classroom use.
The discussion then moved on to standardizing types of peer review processes for open textbooks, perhaps by using markers or badges to transparently delineate which processes were employed on a given text. Billy Meinke from the University of Hawaii shared the Mozilla Science Contributorship Badges and noted that ideally peer review badges would exist both for content and for contributors. This would serve the dual purpose of giving recognition to reviewers along with credit they can use in their tenure and promotion process.
Billy said that Mozilla Science Badges were linked to ORCID ids. Hugh said Rebus will be looking for ways to do the same. Karen added that the markers ought to be non-hierarchical, but still provide adequate recognition.
Nicholas Persa from UW-Madison acknowledged the need for one portal of access that enables and encourages community review – a model that Rebus is hoping to build.
Participants agreed on many topics, such as the need to incentivize reviewers either with payment or recognition, the need for post-publication review in classrooms and continual updating of open textbooks, and badges or markers to validate both content and reviewers. However, a number of questions remained unanswered: what specific value can we offer reviewers, particularly when monetary payment is not an option? Should pre-publication review be done before copyediting? How do we begin making these markers and defining levels of completion for peer review?
Universities such as The University of British Columbia seem to be making strides in recognizing the importance of OERs on tenure committees, and as this spreads to other institutions, there will be a growing need to develop best practices and standards for processes like reviewing.
The Rebus Working Group will continue to work on these and other issues pertaining to peer review. Next steps will be posted on the forum. Join the conversation on the Rebus Community forum thread as we try to figure it all out!
At Rebus, we think one of the best ways to produce open textbooks is by involving students in the process.
That’s why we’re really excited about the handbook we have in development, A Guide to Making Open Textbooks With Students. The book is designed to help faculty find or create projects involving students in the creation of open textbooks or open resources.
The guide will contain:
an overview of open pedagogy – what it is, why it’s valuable, learning objectives and outcomes
resources and assignments for faculty wanting to adopt open pedagogy in their classrooms
case studies of projects in which faculty and students collaborated to create open textbooks and similar open educational resources
student perspectives on their experiences working on such projects
tools you can use for open textbook projects, such as licensing guides, student agreements and more.
Currently our focus is on finding students who have worked on open textbook projects to write 300- to 500-word narratives about their experiences.
If you know of someone who has worked on such a project and might be willing to contribute such a sidebar, please sign up to the forum and reply to this post, or encourage them to do so.
We also still welcome contributions from faculty, so if you have an idea for a resource that you can contribute, we’re all ears–reply here.
One of the pilot open textbook projects the Rebus Community is supporting is Media Innovation & Entrepreneurship, designed to be used in media entrepreneurship courses or other journalism, mass communication or media courses in which media entrepreneurship is taught.
Each “chapter” of the book will be a module that can accompany a week’s lesson in a 15-week course. The work will be designed to be used as a whole, or as standalone modules.
We’ve been reaching out to chapter contributors for the past month or so, and are now only five chapters away from having every chapter in this open textbook project spoken for. Can you fill, or help us find someone to fill, the last major holes?
After these chapters are claimed, we’ll be looking for professionals to contribute sidebars with firsthand experience and faculty to contribute classroom activities. A future edition will also include companion resources for instructors.
Guest Speakers: Josie Gray, BCcampus; Jess Mitchell, OCAD University; Michelle Reed, University of Texas at Arlington Libraries; Krista Greear, University of Washington
What are the best practices to ensure accessibility in open textbooks? In this session, we will talk about methods to ensure accessibility during authoring and post-authoring processes. We’ll also discuss how to audit the accessibility of existing open textbooks.
Are you a Spanish language literature professor looking for an open pedagogy project for your classroom? We’re looking for contributors to expand the Antología Abierta de Literatura Hispánica and you and your students could help! Read more about the project below, then head here to participate.
Here at the Rebus Community, we’ve found working with students to be one of the most interesting approaches to creating open textbooks out there.
We’re pleased to now add another great example of open pedagogy to our stable of projects, with the Antología Abierta de Literatura Hispánica (Open Anthology of Hispanic Literature, AALH) led by Julie Ward of the University of Oklahoma.
The AALH is a collection of public-domain texts from the Hispanic world, with critical introductions and annotations by undergraduate students in Julie’s Introduction to Hispanic Literature and Culture course at the University of Oklahoma. The AALH is intended as a freely accessible digital resource for students of Hispanic literature, and proposes an inclusive, broad, and evolving definition of the canon.
To continue Julie and her students’ work, we are looking for collaborators who will implement the critical edition assignment in their own courses and share the student-created critical editions for inclusion in future editions.
Resources will be offered to support the implementation, including:
Suggested assignment timeline
List of possible texts in public domain
Sample MOU for students
Guide to Creative Commons licensing for students
Community support from others running similar assignments
These resources are offered as a starting point only, and can be adapted to meet your course’s requirements.
If you are teaching a Hispanic literature course at any level and want to work with your students to expand the anthology, head to the project page in the forum, sign up, and let us know you’re interested!
We are looking for chapter reviewers for The Science of Human Nutrition, a new open nutrition textbook from the University of Hawaii.
Guided by OER technologist Billy Meinke, three UoH authors are working on adapting several existing open resources to create a textbook is aimed at high-enrollment undergraduate (100 level) courses in FSHN (Food Science and Human Nutrition). It will cover elementary aspects of several biological sciences, including information on what nutrients are and what nutrients and foods do for humans; how healthy people can best get the amounts of nutrients and foods they need throughout their lifetime; how people and the environment change foods and their nutrient content; and nutrition issues of current interest.
The first six chapters have been drafted and are ready for review. We are seeking volunteers with some experience in nutrition, biology, nursing or other biological sciences to read, review and comment on at least one chapter, in line with some simple review guidelines. Reviewers will be given access to a shared drive with the content and guidelines.
If you’d like to be involved, please head over to the project page on the forum, sign up and let us know!
Here’s a preview of a few of the chapters up for grabs:
At the Rebus Community, we see peer review as a critical part of publishing open textbooks. In particular, it plays an important role in encouraging adoptions – both by assuring those looking to adopt a book that the material is of high quality, and also by building an engaged pool of reviewers who are themselves potential adopters.
Recognising this importance, we are working to develop – with community input – a clear, robust peer review process that can be applied to all open textbooks produced with Rebus (and potentially beyond).
Our recent Office Hours session on Peer Review for Open Textbooks surfaced some of the issues we will seek to address with the working group. These include:
What should pre- & post-production review processes for open textbooks look like?
How can we enable faculty adopters & students to provide feedback directly to authors?
How can we leverage the peer review process to market the book to potential adopters?
How do we manage the concerns and uncertainty around any non-traditional aspects of the review process?
How might reviewers be compensated for their contributions?
What tools do we need to support the process?
If you would like to be part of this group, please visit the project page and let us know you’re interested!
Allison Brown, SUNY Geneseo; Rebel Cummings-Sauls, Kansas State University; Dianna Fisher, Oregon State; Billy Meinke, University of Hawaii; and Anthony Palmiotto, OpenStax
Is there a typical process and timeline for producing an open textbook? Where are the pain points? How could the process be made more efficient? How are staff and faculty working on these projects managing the tasks and timelines involved? How is progress tracked and momentum sustained?