The criteria used to evaluate and select open educational resources and textbooks should be almost the same as for traditional textbooks. The following resources can provide useful information to evaluate open textbooks for possible course adoption.
Altman, William S., Kristen Ericksen, and Judith B. Pena-Shaff. An Inclusive Process for Departmental Textbook Selection. Teaching Of Psychology 33, no. 4, p. 228-231. This article discusses a process for selecting a textbook for lower level Psychology courses, but the questions they ask could be useful when evaluating open textbooks or resources for a class.
This website from Achieve.org provides a rubric to help schools choose open educational resources for their school system. Some of the items in the rubric could be applied to choosing open educational resources and textbooks for a college course.