Category: Working Groups

Peer Review Working Group: Next Steps

Peer review is critical for the acceptance of open textbooks by the wider world. Perceptions of uncertain quality of open textbooks is often cited as a barrier to adoption. Good, transparent review processes will help solve this problem.

While many open textbooks are comprehensively reviewed, there is currently no accepted standard or process for review.

We’ve put together a Peer Review Working Group to help the Open Textbook community come together to develop a clear approach to review. (Would you like to join us? Go here!)

We’ve currently got around 20 members in this group, from as many institutions, including: the Open Textbook Network, OpenStax, SUNY, Oregon State, and Ryerson University. You can find a recap of our first meeting (including the full video) here.

During this meeting, we identified a number of challenges for reviewing Open Textbooks, and our group hopes to provide some solutions, with subgroups focusing on the following issues.

Please join us if you think you can help us figure out smart approaches to reviewing Open Textbooks.

Group 1: Open Textbook Review Board
The Rebus Community has proposed to create a review board that can been leveraged to help find reviewers for open textbooks. Members of the board will be contacted when a book becomes available for review and make efforts within and possibly beyond their institutions to find volunteers. This group will define and help to launch the board. Join Group 1 here.

Group 2: Defining Categories of Review
This group will work to define the purpose and structure of the different kinds of review that could be valuable to an open textbook. They will also help guide the development of dedicated tools and resources for each kind of review. Join Group 2 here.

Group 3: Environmental Scan of Peer Review Tools, Processes & Research
This group will conduct a scan of the existing peer review market in order to inform the development of new review processes. Join Group 3 here.

Group 4: Creating Standard Markers for OT Review
This group will discuss creating a standardised system to clearly identify the kinds of review that any given textbook has undergone. This will link to the work of group 2, who are defining different categories of review. Join Group 4 here.

Group 5: Recognition for Reviewers
This group will discuss ideas of how to recognise and reward reviewer contributions. This may eventually feed into the creation of something similar to the Mozilla Contributorship Badges project, recognising all kinds of participation in open textbook projects, including peer review. Join Group 5 here.

Signup for the groups will stay open, but we encourage you to join by May 20 so we can start moving forward on each of them.

Peer Review Working Group Recap

Peer review is an integral component of traditional academic publishing, but in the world of open textbooks, there is currently no common standard for reviewing in open textbooks. To address this need, Rebus Community has formed a working group to explore peer review for open textbooks. The group held its first meeting May 3 to discuss how the community of open textbook practitioners could build a coherent strategy and best practices around peer review for open textbooks.

Below is a recap of the meeting. You can read next steps and how to get involved here.

Deb Quentel from CALI started the session by describing CALI’s editorial review process for the lessons and textbooks they produce. Deb said CALI has an editorial review board consisting of potential reviewers in specific subject areas that she reaches out to first when looking for reviewers. If no one has the relevant expertise, she asks board members to leverage their networks to find appropriate reviewers. If that fails, the author may suggest potential peers as reviewers. Finally, Deb scouts reviewers online from relevant websites and publications.

In CALI’s peer review process, reviewers are asked to look at subject matter accuracy and whether the content covers the terrain to be expected. Reviewers also evaluate the sophistication of the end-of-chapter questions that are included for students. Reviewers provide a no-more-than-two-page memo of feedback. At CALI, peer reviewers are anonymous and usually review one chapter of a text. About 75% of the chapters for any given book get reviewed. Deb outlines her process in further detail here.

Next, Hugh McGuire of Rebus Community outlined various types of pre- and post-publication review that he hypothesized may need to be done to an open textbook:

  • Subject matter expert review (a formal, pre-publication review, similar to traditional peer review)
  • Open review, in which anyone can make comments (less formal)
  • Student and faculty feedback on a published, beta version of the textbook
  • Ongoing feedback (how does feedback get back to the authors following publication?)
  • Revision and updates (especially for subjects that change frequently)

Karen Lauritsen from the Open Textbook Network gave an overview of OTN’s review process. All new OTN member organizations are able to send faculty to workshops as part of their professional development in open textbooks. In return, they are required to provide a light review for one book in the Open Textbook Library and are given a rubric from which to do so. Karen said that roughly 40% of these reviewers end up adopting a textbook from the library for classroom use.

The discussion then moved on to standardizing types of peer review processes for open textbooks, perhaps by using markers or badges to transparently delineate which processes were employed on a given text. Billy Meinke from the University of Hawaii shared the Mozilla Science Contributorship Badges and noted that ideally peer review badges would exist both for content and for contributors. This would serve the dual purpose of giving recognition to reviewers along with credit they can use in their tenure and promotion process.

Billy said that Mozilla Science Badges were linked to ORCID ids. Hugh said Rebus will be looking for ways to do the same. Karen added that the markers ought to be non-hierarchical, but still provide adequate recognition.

Nicholas Persa from UW-Madison acknowledged the need for one portal of access that enables and encourages community review – a model that Rebus is hoping to build.

Participants agreed on many topics, such as the need to incentivize reviewers either with payment or recognition, the need for post-publication review in classrooms and continual updating of open textbooks, and badges or markers to validate both content and reviewers. However, a number of questions remained unanswered: what specific value can we offer reviewers, particularly when monetary payment is not an option? Should pre-publication review be done before copyediting? How do we begin making these markers and defining levels of completion for peer review?

Universities such as The University of British Columbia seem to be making strides in recognizing the importance of OERs on tenure committees, and as this spreads to other institutions, there will be a growing need to develop best practices and standards for processes like reviewing.

The Rebus Working Group will continue to work on these and other issues pertaining to peer review. Next steps will be posted on the forum. Join the conversation on the Rebus Community forum thread as we try to figure it all out!

You can watch the video recap here.

Additional Resources:

Making Open Textbooks Accessible: A Test Case

The Rebus Community is pleased to announce the latest addition to our stable of projects: Financial Strategy for Public Managers.

Led by Justin Marlowe of the University of Washington, this text is targeted at Master of Public Administration students, but could also serve as the core text for a range of undergraduate, graduate and professional education courses on public financial management.

In addition to being a great contribution to students and faculty in the area, this project will be serving as a special test case for the Rebus approach to accessibility. We understand the importance of accessibility and inclusive design when it comes to both the production and consumption of open textbooks and are working to ensure appropriate checks and tools are built into our processes (read more about our approach).

If you can help with reviewing, proofreading or promotion, you can sign up at the Financial Strategy project post. Or, if you can contribute to our work on accessibility, head over to the forum and add your name to the working group we’re forming.

The Rebus Approach to Accessibility & Inclusivity

In part one of our series on accessibility, we covered what accessibility is and why it’s important. Now, learn about what we’re doing to ensure accessibility for the open textbooks we support.

How Do We Ensure Accessibility & Inclusivity of the Open Textbooks Created with Rebus Support?

The Rebus Community is committed to ensuring that all Open Textbooks coming through the Rebus process go through (and pass!) an accessibility audit, and indeed that we build accessibility right into the authoring process.

Currently we are using an in-progress open textbook, Financial Planning in Public Policy, as a pilot/test case for developing an accessibility audit on Rebus Community supported books.

The idea behind what we are proposing is to:

  1. make sure that authors making Open Textbooks are aware of good accessibility practice from the start of the process
  2. help them easily implement good accessibility practice
  3. provide a standard accessibility audit process after an Open Textbook is created (possibly with some tools associated)
  4. provide a mechanism to “fix” accessibility problems found in the audit process
  5. have an “accessibility stamp of approval” for projects that have successfully passed the accessibility audit

Collaborators will be developing an accessibility checklist, which we will then apply to the book, along with figuring out next steps for what such a process should look like.

This process is being undertaken in partnership with the Inclusive Design Research Center at OCAD University and the University of Washington. Follow updates on this topic on the project post in the Rebus Community forum.

For more on accessibility and universal design, check out these resources:

Pressbooks EDU Guide chapter on Accessibility and Universal Design
BC Campus Accessibility Toolkit

What is Accessibility?

At Rebus, we’re committed to ensuring that all Open Textbooks coming through the Rebus process are accessible. In fact, we are working on building accessibility right into the authoring process.

What do we mean by accessibility?

Accessibility is the term used for, roughly, “making it easy for people with disabilities (say people with visual impairments, people with learning disabilities, among others) to access content.” See:

https://en.wikipedia.org/wiki/Web_accessibility

In general the web is good at this … there is markup “behind” the text that you see on a webpage, and that markup (if done correctly) is semantic, meaning it tells you things about the kind of content you are reading. (This is not the case with, for example, a paper book, many PDFs, and some EPUBs).

For instance, in Pressbooks, you get markup that looks something like:

<h1 class=“chapter-title”>Chapter  Title</h1>

<h2>Section title</h2>

Some content.

<h3>Subsection title</h3>

More content.

Usually that semantic markup is translated into visual styling (bold text, etc.), so that a reader can distinguish different chapter/section levels, and this information is processed “automatically” as part of reading.

In the case, for instance, of a visually impaired user of content, who is using a screen reader (software that “reads” the text out loud to the user), the fact that the content uses semantic markup means that the screen reader tool “knows” that h1 is  a chapter title, h2 is a section title etc.

Another common use case is “alt tags” on images … which in good accessibility practice can/should describe the image so that someone using a screen reader can be told what is in the image automatically, even if they cannot see it.

Next: Read about what we’re doing to ensure the accessibility of open textbooks created with Rebus support.