Peer Review Working Group Recap

Peer review is an integral component of traditional academic publishing, but in the world of open textbooks, there is currently no common standard for reviewing in open textbooks. To address this need, Rebus Community has formed a working group to explore peer review for open textbooks. The group held its first meeting May 3 to discuss how the community of open textbook practitioners could build a coherent strategy and best practices around peer review for open textbooks.

Below is a recap of the meeting. You can read next steps and how to get involved here.

Deb Quentel from CALI started the session by describing CALI’s editorial review process for the lessons and textbooks they produce. Deb said CALI has an editorial review board consisting of potential reviewers in specific subject areas that she reaches out to first when looking for reviewers. If no one has the relevant expertise, she asks board members to leverage their networks to find appropriate reviewers. If that fails, the author may suggest potential peers as reviewers. Finally, Deb scouts reviewers online from relevant websites and publications.

In CALI’s peer review process, reviewers are asked to look at subject matter accuracy and whether the content covers the terrain to be expected. Reviewers also evaluate the sophistication of the end-of-chapter questions that are included for students. Reviewers provide a no-more-than-two-page memo of feedback. At CALI, peer reviewers are anonymous and usually review one chapter of a text. About 75% of the chapters for any given book get reviewed. Deb outlines her process in further detail here.

Next, Hugh McGuire of Rebus Community outlined various types of pre- and post-publication review that he hypothesized may need to be done to an open textbook:

  • Subject matter expert review (a formal, pre-publication review, similar to traditional peer review)
  • Open review, in which anyone can make comments (less formal)
  • Student and faculty feedback on a published, beta version of the textbook
  • Ongoing feedback (how does feedback get back to the authors following publication?)
  • Revision and updates (especially for subjects that change frequently)

Karen Lauritsen from the Open Textbook Network gave an overview of OTN’s review process. All new OTN member organizations are able to send faculty to workshops as part of their professional development in open textbooks. In return, they are required to provide a light review for one book in the Open Textbook Library and are given a rubric from which to do so. Karen said that roughly 40% of these reviewers end up adopting a textbook from the library for classroom use.

The discussion then moved on to standardizing types of peer review processes for open textbooks, perhaps by using markers or badges to transparently delineate which processes were employed on a given text. Billy Meinke from the University of Hawaii shared the Mozilla Science Contributorship Badges and noted that ideally peer review badges would exist both for content and for contributors. This would serve the dual purpose of giving recognition to reviewers along with credit they can use in their tenure and promotion process.

Billy said that Mozilla Science Badges were linked to ORCID ids. Hugh said Rebus will be looking for ways to do the same. Karen added that the markers ought to be non-hierarchical, but still provide adequate recognition.

Nicholas Persa from UW-Madison acknowledged the need for one portal of access that enables and encourages community review – a model that Rebus is hoping to build.

Participants agreed on many topics, such as the need to incentivize reviewers either with payment or recognition, the need for post-publication review in classrooms and continual updating of open textbooks, and badges or markers to validate both content and reviewers. However, a number of questions remained unanswered: what specific value can we offer reviewers, particularly when monetary payment is not an option? Should pre-publication review be done before copyediting? How do we begin making these markers and defining levels of completion for peer review?

Universities such as The University of British Columbia seem to be making strides in recognizing the importance of OERs on tenure committees, and as this spreads to other institutions, there will be a growing need to develop best practices and standards for processes like reviewing.

The Rebus Working Group will continue to work on these and other issues pertaining to peer review. Next steps will be posted on the forum. Join the conversation on the Rebus Community forum thread as we try to figure it all out!

You can watch the video recap here.

Additional Resources:

Student Contributors Wanted: Guide to Making Open Textbooks With Students

At Rebus, we think one of the best ways to produce open textbooks is by involving students in the process.

That’s why we’re really excited about the handbook we have in development, A Guide to Making Open Textbooks With Students. The book is designed to help faculty find or create projects involving students in the creation of open textbooks or open resources.

The guide will contain:

  • an overview of open pedagogy – what it is, why it’s valuable, learning objectives and outcomes
  • resources and assignments for faculty wanting to adopt open pedagogy in their classrooms
  • case studies of projects in which faculty and students collaborated  to create open textbooks and similar open educational resources
  • student perspectives on their experiences working on such projects
  • tools you can use for open textbook projects, such as licensing guides, student agreements and more.

Currently our focus is on finding students who have worked on open textbook projects to write 300- to 500-word narratives about their experiences.

If you know of someone who has worked on such a project and might be willing to contribute such a sidebar, please sign up to the forum and reply to this post, or encourage them to do so. 

We also still welcome contributions from faculty, so if you have an idea for a resource that you can contribute, we’re all ears–reply here.

Making Open Textbooks With Students image

Five More Chapter Authors Needed for Media Innovation & Entrepreneurship Open Textbook

One of the pilot open textbook projects the Rebus Community is supporting is Media Innovation & Entrepreneurship, designed to be used in media entrepreneurship courses or other journalism, mass communication or media courses in which media entrepreneurship is taught.

Each “chapter” of the book will be a module that can accompany a week’s lesson in a 15-week course. The work will be designed to be used as a whole, or as standalone modules.

We’ve been reaching out to chapter contributors for the past month or so, and are now only five chapters away from having every chapter in this open textbook project spoken for. Can you fill, or help us find someone to fill, the last major holes?  

The final topics in need of a chapter author are:

  • Ideation
  • Project Management Skills & Technologies
  • Human-centered Design
  • Competitive Analysis
  • UX/UI Testing and Iterating

If you can contribute (or have someone to suggest we reach out to), please sign up to the Rebus Community Forum and reply to this post.

After these chapters are claimed, we’ll be looking for professionals to contribute sidebars with firsthand experience and faculty to contribute classroom activities. A future edition will also include companion resources for instructors.

You can view the table of contents in development here and learn more about the project here.

Dr. Michelle Ferrier of Ohio University is the lead author on the project, with assistance from Rebus’ own Liz Mays, who is also a faculty associate at Arizona State University.

The project is planned for beta release and student use and testing in August 2017.

media-innovation-book-image

June Office Hours: Accessibility in Open Textbooks

Monday, June 5, at 3 p.m. EST

Topic: Accessibility in Open Textbooks

Guest Speakers: Josie Gray, BCcampus; Jess Mitchell, OCAD University; Michelle Reed, University of Texas at Arlington Libraries; Krista Greear, University of Washington

What are the best practices to ensure accessibility in open textbooks? In this session, we will talk about methods to ensure accessibility during authoring and post-authoring processes. We’ll also discuss how to audit the accessibility of existing open textbooks.

RSVP for the session. Click https://zoom.us/j/274741128 to join the session day of. (Note that the session will be recorded.)

If you have any questions, or have difficulty entering the call, email us at contact@rebus.community.

Office Hours Accessibility Image

Open Pedagogy in Action: Antología Abierta de Literatura Hispánica

Are you a Spanish language literature professor looking for an open pedagogy project for your classroom? We’re looking for contributors to expand the Antología Abierta de Literatura Hispánica and you and your students could help! Read more about the project below, then head here to participate.


Here at the Rebus Community, we’ve found working with students to be one of the most interesting approaches to creating open textbooks out there.

This approach was the driving force behind the first iteration of our Open Anthology of Earlier American Literature project, which is now being expanded by lead editor Tim Robbins. Our January Office Hours session also discussed the issue, and from that session we began work on a guide to working with students on open textbooks.

We’re pleased to now add another great example of open pedagogy to our stable of projects, with the Antología Abierta de Literatura Hispánica (Open Anthology of Hispanic Literature, AALH) led by Julie Ward of the University of Oklahoma.

The AALH is a collection of public-domain texts from the Hispanic world, with critical introductions and annotations by undergraduate students in Julie’s Introduction to Hispanic Literature and Culture course at the University of Oklahoma. The AALH is intended as a freely accessible digital resource for students of Hispanic literature, and proposes an inclusive, broad, and evolving definition of the canon.

To continue Julie and her students’ work, we are looking for collaborators who will implement the critical edition assignment in their own courses and share the student-created critical editions for inclusion in future editions.

Resources will be offered to support the implementation, including:

  • Assignment brief
  • Marking rubric
  • Suggested assignment timeline
  • List of possible texts in public domain
  • Sample MOU for students
  • Guide to Creative Commons licensing for students
  • Community support from others running similar assignments

These resources are offered as a starting point only, and can be adapted to meet your course’s requirements.

If you are teaching a Hispanic literature course at any level and want to work with your students to expand the anthology, head to the project page in the forum, sign up, and let us know you’re interested!

Call for Reviewers: The Science of Human Nutrition

We are looking for chapter reviewers for The Science of Human Nutrition, a new open nutrition textbook from the University of Hawaii.

Guided by OER technologist Billy Meinke, three UoH authors are working on adapting several existing open resources to create a textbook is aimed at high-enrollment undergraduate (100 level) courses in FSHN (Food Science and Human Nutrition). It will cover elementary aspects of several biological sciences, including information on what nutrients are and what nutrients and foods do for humans; how healthy people can best get the amounts of nutrients and foods they need throughout their lifetime; how people and the environment change foods and their nutrient content; and nutrition issues of current interest.

The first six chapters have been drafted and are ready for review. We are seeking volunteers with some experience in nutrition, biology, nursing or other biological sciences to read, review and comment on at least one chapter, in line with some simple review guidelines. Reviewers will be given access to a shared drive with the content and guidelines.

If you’d like to be involved, please head over to the project page on the forum, sign up and let us know!

Here’s a preview of a few of the chapters up for grabs:

 

Peer Review Working Group & Office Hours Recap

At the Rebus Community, we see peer review as a critical part of publishing open textbooks. In particular, it plays an important role in encouraging adoptions – both by assuring those looking to adopt a book that the material is of high quality, and also by building an engaged pool of reviewers who are themselves potential adopters.

Recognising this importance, we are working to develop – with community input – a clear, robust peer review process that can be applied to all open textbooks produced with Rebus (and potentially beyond).

Our recent Office Hours session on Peer Review for Open Textbooks  surfaced some of the issues we will seek to address with the working group. These include:

  • What should pre- & post-production review processes for open textbooks look like?
  • How can we enable faculty adopters & students to provide feedback directly to authors?
  • How can we leverage the peer review process to market the book to potential adopters?
  • How do we manage the concerns and uncertainty around any non-traditional aspects of the review process?
  • How might reviewers be compensated for their contributions?
  • What tools do we need to support the process?

If you would like to be part of this group, please visit the project page and let us know you’re interested!

You can read a summary of the Office Hours session or watch the recording below.

Next Office Hours: OER Workflows for Open Textbooks

Monday, May 1, at 3 p.m. EST

Topic: OER Workflows for Open Textbooks

Guest Speakers:

Allison Brown, SUNY Geneseo; Rebel Cummings-Sauls, Kansas State University; Dianna Fisher, Oregon State; Billy Meinke, University of Hawaii; and Anthony Palmiotto, OpenStax

Is there a typical process and timeline for producing an open textbook? Where are the pain points? How could the process be made more efficient? How are staff and faculty working on these projects managing the tasks and timelines involved? How is progress tracked and momentum sustained?

RSVP. Click https://zoom.us/j/950787351 to join the session day of.

(Note that the session will be recorded.)

If you have any questions, email us at contact@rebus.community.

Office Hours Title Slide

Office Hours Recap: Peer Review for Open Textbooks

In this month’s Rebus Community & Open Textbook Network Office Hours we were joined by special guests to discuss peer review for open textbooks.

Anthony Palmiotto, Editorial Director at OpenStax, discussed the OpenStax approach to peer review, the first part of which follows a traditional in-development review model, similar to that of the major commercial publishers. During this process, they work to ensure a mix of different types of reviewers, profiles and representation of different regions, especially for disciplines where geographic and political balance is important. The second part of their review process involves a post-development feedback mechanism where anyone can submit feedback on a book. This feedback is then reviewed by faculty, with minor revisions being made monthly, and major revisions yearly. OpenStax aim to be as transparent as possible with these processes, listing reviewers at the beginning of each book, which also helps to establish credibility.

Karen Lauritsen of the Open Textbook Network shared the OTN review process, in which faculty who attend an OTN workshop are asked to complete a review of a textbook of their choice in the Open Textbook Library. Karen indicated that this process reassures faculty of the quality of available open textbooks, and encourages adoptions. She also noted that because the Library brings together texts from a wide range of sources, they have all undergone differing levels and methods of pre-production review. This highlights the current variability of open textbook review processes. Later in the session it was suggested that there might be an opportunity to standardise this in such a way that is useful for everyone working on open textbooks, similar to the Creative Commons licenses (i.e. providing a shorthand indicator of the type/level of review done to a book).

Deb Quentel from CALI (The Center for Computer Assisted Legal Instruction) shared her approach to finding reviewers, which is to call on authors for suggestions first, then on CALI’s editorial board of about 60 faculty and librarians, all with different areas of expertise. Deb gave some great advice for finding ‘strangers’ to bring in as reviewers, looking online for those writing and teaching in related areas. Once found, reviewers are given some guideline questions and two weeks to produce a 1-2 page memo, and are offered a small honorarium in compensation. Deb also mentioned that CALI often asks subject matter experts to proofread chapters as well, adding a second layer of review that can catch other errors or make additional suggestions.

Jon Udell joined us from Hypothesis and gave a short demonstration of how the current Hypothesis annotation tools can be used to review content. He also indicated a strong interest in engaging with the Community and finding test cases for Hypothesis’ current and future tools, so that those using them can be involved with the development.

Hugh McGuire of the Rebus Community capped off the session by framing the Rebus approach to the question of how best to manage peer review for open textbooks. He reiterated the Rebus objective to develop community-driven processes, and leveraging collaboration as the key to making open textbooks successful at scale. He discussed the different kinds of review we might expect to see develop, from a formal/traditional peer review, to an open review later in the publication process, to ongoing feedback from users of the book to the author. Last, he closed with an invitation to all who are interested in the topic to help us figure out what this should all look like by joining a working group.

The discussion continued on once the speakers wrapped up, touching on compensation, building community around books, the importance of transparent processes and the possibility of creating standardised measures or levels of peer review, similar to the Creative Commons license, where there’s a widely recognised ‘badge’ as a shorthand. The working group will pursue these questions and ideas and we encourage you to join us!

You can see the full video of the session here:

Changes to the Community Forum

Here at Rebus we’re always reflecting on how we do things, and changing as we learn. In light of this ever-evolving approach, we’ve recently made a couple of changes to the forum. These changes reflect how we have found the Community uses the forum since we launched, and will hopefully be more useful for those visiting our shores.

The biggest change is that we have two new categories replacing one that was intended for discussions about managing open textbook programs. Instead, we have added Projects: Working Groups which will house the single-issue projects/discussions we’re looking to foster, including our accessibility and peer review working groups. The second new category is a General Discussion where anyone in the community can share their experiences, challenges, ideas or interesting tidbits! We look forward to seeing you there.