Contributed papers will be presented on Sunday, Oct. 6 in two sessions: 1:00 p.m. – 2:00 p.m. and 3:00 p.m. – 4:00 p.m.
Session 2: 3:00 p.m. – 4:00 p.m.
|Milwaukee Room, 3:00 p.m. – 4:00 p.m.|
|3:00-3:18||#10. Flipping the Script: Assessing an Online Approach to Training Student Workers|
|3:20-3:38||#11. Make It a Game! Engaging Students in Searching the Literature|
|3:40-3:58||#12. Developing an Interactive Tool to Assist in Complying with the NIH Public Access Policy.|
Executive AB, 3:00 p.m. – 4:00 p.m.
|3:00-3:18||#13. Doing better with library classes in 2019: Streamlining scheduling, marketing, and data gathering|
|3:20-3:38||#14. How are researchers documenting their day-to-day activities?|
Paper #1: Still feeling the effects: a citation analysis of the highly cited, retracted article on MMR vaccines and autism
Winner of Best Research Paper Award
Authors: Elizabeth Suelzer, Jennifer Deal, Karen Hanus, Barbara Ruggeri, Rita Sieracki, Elizabeth Witkowski; Medical College of Wisconsin Libraries, Advocate Aurora, Carroll University
Introduction: In 1998, Andrew Wakefield published an article that allegedly showed a causal relationship between the measles, mumps and rubella (MMR) vaccination and autism. Although this article was twice retracted (Murch et al., 2004; The Editors of The, 2010), and questions arose about the credibility of Wakefield’s data (Deer, 2011) the infamous article continues to be highly cited. This research project analyzed the cited by references of the 1998 Wakefield article to learn more about who cited it and how the article was cited.
Methods: We conducted a cited reference search in Web of Science to identify literature that cited the 1998 Wakefield article. The citations and full text copy for each reference were uploaded into Covidence where each reference was screened to determine the characteristic of the in-text citations using an established taxonomy (Bornmann & Daniel, 2008; Leung et al, 2017) and to see if the retractions of the Wakefield article were referenced. The screening was blinded, and conflicts were resolved by group consensus.
Results: A total of 1,153 citations were analyzed. Articles were not included in this study if they were not published in English or if we were unable to find the Wakefield reference in the article text. The most common citation characteristics were negational (72%), perfunctory (9%) and affirmational (8%). The Wakefield article was retracted twice: a partial retraction by 10 of the 12 authors in 2004, and a full retraction in 2010. The partial retraction was referenced in 32% of the articles published between 2004-2009. Retractions were referenced in 71% of the articles published after 2010. Since 2013, the percentage of authors who referenced the retraction is continually growing.
Conclusions: Overwhelmingly, scholarly literature negated the findings of the 1998 Wakefield article, even before the article was retracted. Many citing authors pointed out the methodological flaws of the study and the small sample size. Highly cited papers are presumed to hold more scholarly weight than less cited papers, however, this article was cited for all the wrong reasons.
A significant number of articles published after the retraction did not cite the retraction. Although most citation styles have recommendations on how to cite retractions, not all authors follow the recommendations. There is a need for greater vigilance in ensuring that retracted articles are referenced properly.
Paper #2: Letting students struggle: Pros and cons of letting students struggle in Problem Based Learning(PBL)
Author: Charlotte Beyer; Rosalind Franklin University of Medicine and Science
Objective: To describe a librarian’s experience of designing a Problem Based Learning (PBL) activity for medical students in a pre-clerkship clinical skills course. The librarian will outline successes, challenges, and lessons learned.
Methods: In the summer of 2018, the library lecture for the clinical skills course was re-designed in a more problem-based learning (PBL) format for point of care resources. However instead of introducing content in a presentation format as previously done, the learners directed learning with librarian as a guide when needed. In the summer of 2018, the activity was piloted with 60 rising third year medical students and five fourth year students as group leaders. In 2019, the activity went through a slight re-design with all 190 students and the course director as a guide with two librarians.
Results: One of the major challenges of letting students struggle in PBL is the idea of discomfort during learning. This impacts both librarians and students. Librarians needed to be able to adapt to the learner needs, and students needed to understand discomfort is part of learning and clinical practice. Once both sides understood this, the activity was better designed, and the students asked deeper and more meaningful questions. Students commented they got much more out of the activity than a traditional lecture, even if some did not like the feeling of not knowing every answer.
Conclusions: One of the major challenges with PBL is the instructor such as a librarian has less control of learning during the actual workshop, which is why solid design of the activities prior to sessions are so important. Librarians also need to account for students not being okay with being uncomfortable with encouragement and tailored guidance.
Paper #3: Doing better with LibGuides in 2019: Creating a departmental LibGuide template for an evolving library
Authors: Julia C. Stumpff; Jason A. Lilly; Indiana University School of Medicine, Ruth Lilly Medical Library
Objective: In the spring of 2018, a campus wide LibGuides Task Force initiated a peer-review process to update and review all guides by the summer of 2019. However, many of the Ruth Lilly Medical Library librarians who originally created departmental LibGuides no longer worked at the library. Without guidance from creators and no formula to follow, librarians who inherited responsibility for guides were overwhelmed by the task of updating them. To address this issue and to attain compliance with institutional standards, a departmental LibGuide template was created.
Methods: Two librarians looked at other health science libraries’ LibGuides, studied institutional guidance for LibGuides, and researched best practices for LibGuide design and for writing on the web. Further, they analyzed existing departmental Libguides to determine most-used page headings. Based on this research and analysis, the librarians created a departmental LibGuide template. This template was presented to librarians, and changes were made in response to feedback received.
Results: Since initial implementation, the template has been tweaked in response to departmental feedback and additional input from librarians. The responses from librarians have been positive, and it is anticipated that once all guides are updated, future Libguide updates will not be as cumbersome.
Conclusions: It is hoped that using a LibGuide template will make it easier for librarians to consistently maintain departmental LibGuides. Future plans include creating an asset list of ebooks to streamline the process further and conducting usability research in order to assess the effectiveness of the template design.
Paper #4: Persistence, Creativity, and Resilience: A Bird’s Eye View of Managing an Evidence-based Medicine Curriculum
Authors: Amy Blevins and Laura Menard; Indiana University School of Medicine, Ruth Lilly Medical Library
Objective: In 2016, efforts began at a large multi-campus medical school to integrate EBM throughout all years of the undergraduate medical school curriculum. This created interesting challenges and opportunities for the new associate director (AD) and her team. Over time, the AD began overseeing the EBM curriculum of a large multi-campus medical school, and transforming the library’s educational program. This required taking a bird’s eye view of her team, accreditation standards, medical education trends, and learning theories to approach this large-scale project.
Methods: First steps involved relationship building, information gathering about the school, and focusing on accreditation standards to prepare for an LCME site visit. The AD participated in curriculum committee meetings, LCME mock visits, and established a curriculum team within the library. The curriculum team used knowledge of medical education and EBM standards and trends to frame the ways librarians could best meet the needs of the school of medicine. In addition, the AD led a team of medical school faculty in creating a new EBM “thread” to develop school-level learning objectives.
Results: Librarians are taking on more responsibility for designing and teaching EBM sessions. Library administration developed two new positions to support instructional efforts, assessment and create in-house training opportunities. Course directors and curriculum developers view librarians as partners and leaders in EBM and critical appraisal. This has led to librarians beginning to work in other areas such as data management and translational sciences instruction. Based on the item analysis, students that took the USMLE Step 1 in 2018 were at the national average, and improvement from being below the national average in 2017.
Conclusions: Taking a bird’s eye view has led to increased satisfaction inside and outside of the library. However, there is ongoing work involved with addressing challenges related to proving value and occasionally combating perceptions of librarians overstepping boundaries.
Paper #5: Current Practices in Data Management Education in Nursing Doctoral Programs
AUthors: Rebecca Raszewski, Abigail Goben, Martha Dewey Bergren, Krista L. Jones, Catherine J. Ryan, Alana D. Steffen, Susan C. Vonderheid; University of Illinois at Chicago
Objective: The inclusion of data management (DM) instruction within nursing doctoral curricula has not been systematically examined. The purpose of this study is to identify existing DM educational practices within nursing curricula.
Methods: Surveys were created for DNP and PhD program directors. Questions were based on the stages of the UK Data Service Research Data Lifecycle. Questions for each survey included: how and by whom DM is being taught within the nursing doctoral curricula; the kinds of data students are using for their DNP projects or dissertations; how students’ data are being preserved; and if there are any data sharing policies governing the students’ projects. Over 450 program directors from 338 DNP and PhD programs were contacted. An exemption was received from the institution’s IRB office to conduct the surveys.
Results: One hundred and four (104) nursing doctoral program directors responded. Sixty-seven (67) were from DNP programs while 37 were from PhD program. Although nursing doctoral program directors report that they are teaching stages of the research data life cycle, DM is mostly being taught through individual mentoring or a single lecture within a required course. Almost 50% of all respondents stated they are not requiring nursing doctoral students to do a DM plan. The majority of nursing doctoral program directors indicated that their college or department did not preserve students’ project data nor were they aware of a specific data policy.
Conclusions: Nursing doctoral programs need to develop an awareness of data policies at their college or institution and clarify student project data sharing and ownership post-graduation. Nursing faculty may need additional support for mentoring students in DM. There are opportunities for educational collaboration between librarians and nursing faculty.
Paper #6: It’s Not the Upside Down: Creating a Flipped Classroom Experience for Critical Appraisal
Authors: Laura Menard and Amy Blevins; Indiana University School of Medicine, Ruth Lilly Medical Library
Objective: Critically appraising medical literature is a skill that every medical student needs, however, finding experienced instructors and time in the curriculum can be challenging – especially with multiple campuses. An assistant director worked with an associate director to design a flipped classroom model for teaching critical appraisal skills to medical students. In addition, the assistant director designed an in-house training program and facilitator guides to ensure that librarians had the skills to deliver the sessions to campuses around the state.
Methods: Using Kaltura, the assistant director designed and recorded short video tutorials with embedded formative assessments for prognosis, harm, and diagnosis. Librarians attended a “teach through” where they were introduced to the material. Students completed the video tutorials and a summative quiz within a course management system ahead of a facilitated small group session. The summative quiz included questions to assess satisfaction and comprehension. All librarians reviewed the summative quiz data to customize the sessions.
Results: Summative quiz feedback was positive. The data provided by the embedded formative assessments showed that students were able to answer questions based on the information provided in the tutorials. In addition, anecdotal evidence from the course director shows that students were satisfied with the new modality. Librarians used both the summative and formative quiz results to tailor the sessions to individual campus needs. This also allowed for scaffolding and reinforcement of instruction across the three sessions.
Conclusions: The flipped classroom model allowed librarians to maximize critical appraisal instruction while minimizing face-to-face classroom time. In addition, librarians felt comfortable serving as facilitators rather than lecturers for material that librarians had not previously taught. We will continue to use this model since it provides the students with self-directed learning opportunities with minimal disruption to course and librarian schedules.
Paper #7: Facilitating Open Access through collaboration with clinical departments
Author: Jessica Petrey; University of Louisville
Background: In order to reduce economic barriers to health sciences information, a group of physicians consulted their clinical librarian about the university libraries’ capabilities in supporting their goal of creating and publishing two open access journals. The faculty’s desire was to have these journals not only be free to read, but also free of publication costs to potential authors. A working group including division faculty and staff members, clinical librarian, and library faculty and staff from archives and special collections was created in July of 2016 to explore the potential project.
Description: The work group spent 6 months on negotiations and project planning, including deciding on design aspects of the site, calling for editors, authors, and reviewers, and drafting a memorandum of understanding detailing labor responsibilities between the University Libraries’ and the clinical department. Instead of subscription or submission fees, the project was funded by a combination of grants, departmental support and existing library services. The journals are housed in the University Libraries’ institutional repository platform, but the editorial board and technical support are staffed by clinical faculty and employees.
Conclusion: Since its launch in January 2017, the Journal of Respiratory Infections has published 52 articles which have been downloaded a total of 7890 times in 127 different countries. The 14 articles published in the Journal of Refugee and Global Health, which launched in August 2017, have been downloaded 1342 times in 86 countries. By the end of the session, participants will be able to thoughtfully examine opportunities that may exist within their own institution and service model to facilitate equitable dissemination of information through the support of open access journals.
Paper #8: Expanding Research Data Management Education for Librarians & Health Sciences Affiliates at the University of Minnesota, Duluth
Authors: Lisa McGuire, Caitlin Bakker, Sarah Jane Brown, Katherine Chew, Shanda Hunt;
Bio-Medical Library, University of Minnesota
Purpose: The Expanding Research Data Management Education grant by the Bio-Medical Library, University of Minnesota proposed activities to achieve two primary goals. The first was to enable health science librarians across the Greater Midwest Region (GMR) to build research data management knowledge and skills and to develop actionable steps to provide data management services at their libraries. The second was to enable health science faculty and graduate/professional students at the University of Minnesota Duluth (UMD) to better understand data management best practices, be better positioned to prepare more competitive grant proposals, and learn about the benefits of preparing datasets for preservation, sharing, and re-use.
Setting/Participants/Resources: We offered a half-day continuing education workshop to 21 librarians in August 2018. GMR librarians had the initial opportunity to apply for $1000 travel stipend to attend the training in Minneapolis, MN. Caitlin Bakker presented the session and she organized and moderated a monthly, online Data Management Community of Practice (CoP) to support the ongoing development of librarian data management skills through the duration of the funding period. The CoP ran from September 2018-May 2019.
The second activity was a half-day research data management workshop for students, faculty and staff at the University of Minnesota Duluth. Thirty-two participants from the Medical School, College of Pharmacy and other colleges attended the session in January 2019. Six librarians from the Bio-Medical Library traveled to Duluth to present the content. In addition to the workshop, the following morning four UMD affiliates met with the workshop trainers to consult on various research data management topics.
Results/Outcomes: Our evaluations show that the original project goals and objectives were met. The overwhelming majority of attendees of the ‘Data Management for Librarians’ CE indicated they either ‘strongly agreed’ or ‘somewhat agreed’ with questions asking about their ability to explain key aspects of research data management in the health sciences (21 out of 21 responses), identify opportunities to integrate data services into their existing roles (19 out of 21 responses), ability to compare and contrast different data repositories for sharing and retrieval (17 out of 21 responses), ability to apply best practices to the creation of a data management plan (19 out of 21 responses), and ability to assess a data management plan and suggest ideas for improvement (20 out of 21 responses). There were similar evaluation results reported by the attendees of the event at UMD. Attendees reported that they either ‘strongly agreed’ or ‘somewhat agreed’ with questions asking about their ability to define a research project, the steps involved and identify places in the process where things can break down (24 out of 25 responses), ability to break down the data management process into concrete steps and actions (24 out of 24 responses), acquired a basic understanding of relevant legal, institutional policies (21 out of 25 responses), knowledge of grant requirements for long-term storage (18 out of 21 responses), awareness of subject repositories and journal data policies (24 out of 24 responses), and having a basic understanding of how to plan for data retention and control after completion of a research project (23 out of 25 responses).
Evaluation Method: We used Qualtrics to distribute our evaluation surveys after each activity (librarian CE, UMD workshop, librarian CoP) and asked participants to rank their agreement with various questions via a five-point Likert scale ranging from “strongly agree” to “not applicable.”
Paper #9: A Historical Analysis of the Library’s Role in Nursing Education: The Johns Hopkins Nurses’ Journal Club and Society of Teresians, 1891 – 1925
Author: Jodi Jameson; The University of Toledo, Mulford Health Science Library
Objective: This historical study examines the activities of the Nurses’ Journal Club and Society of Teresians at the Johns Hopkins Hospital (JHH) Training School for Nurses in the late 19th and early 20th centuries. Both groups were linked to the development of the JHH Nursing School Library during a time when formalized nursing education was still relatively new and professional nursing literature was just beginning to blossom. This study aims to provide insight into the very early years of library support for nursing, offering a historical perspective on library information-seeking behavior among nurses and nursing students.
Background: The Nurses’ Journal Club was formed as a required part of nursing students’ education at JHH. Meetings were held in the Nursing School Library and consisted of the reading and analysis of published medical literature. The Society of Teresians was established to provide a platform for the study of nursing history. The Teresians were also instrumental in developing the Historical Collection of the Library by acquiring books, prints and artwork.
Methods: Evidence was analyzed and synthesized from primary archival sources including original manuscripts, documents and letters in the Alan Mason Chesney Medical Archives at the Johns Hopkins Medical Institutions.
Results: The influence of the Journal Club and Teresians positively impacted the early growth of the JHH Nursing School Library. In addition, nursing students at JHH cultivated a rich and lasting enthusiasm for their chosen career through the study of professional literature and nursing history. Several early graduates of the Training School became notable leaders in health care and nursing education.
Conclusion: For health science librarians today, this story serves as a model for how we can creatively and actively engage students in the use of information to support their professional growth and development. This can be accomplished by coordinating and hosting journal club activities or providing resources, instruction and programming on the history of the health sciences.
Paper #10: Flipping the Script: Assessing an Online Approach to Training Student Workers
Author: Laura Menard; Indiana University School of Medicine, Ruth Lilly Medical Library
Objective: To determine whether an online, learning management system-based training program was successful in preparing student workers for everyday tasks.
Methods: The main service desk at a large medical library is staffed largely by student workers. The student workers do the bulk of the circulation and basic reference work. In order to ensure that student workers are adequately prepared to complete everyday tasks, a suite of training modules was developed in Canvas, the university’s learning management system (LMS). The goal of creating this LMS-based training was to ensure that students received a standardized, comprehensive introduction to their job that, once completed, prepared them to begin performing basic reference and circulation tasks at the desk. The Access Services manager worked with his direct supervisor to create a skills and competency-based survey for students to complete after finishing the online training modules to gauge whether or not the new system was accomplishing the goal it was designed to accomplish.
Results: As of June 2019, six new students have been trained and responded to the survey. Initial responses show that the training gives students skillsets that are sufficient to begin performing basic work-related tasks such as searching the catalog, using circulation software, and explaining policies and procedures. Students will continue to develop skills in complex referrals, chat reference systems, and advanced reference. By October of 2019, we anticipate having an n of at least 10 to present.
Conclusions: Based on initial analysis, an online training system seems to provide student workers with a basic set of skills that enable them to perform required tasks at the service desk soon after they begin employment. Next steps will be creating additional modules to attempt to teach students more advanced skills and modifying existing modules to clarify any points of confusion.
Paper #11: Make It a Game! Engaging Students in Searching the Literature
Authors: Carol Shannon; Taubman Health Sciences Library, University of Michigan
Background: In 2015, I revised the library curriculum for the College of Pharmacy, working with faculty to increase the number of sessions from two to three, scaffolding the instruction, and introducing a flipped classroom model, with active learning and assessments throughout. In one session, students were not learning as much as I had hoped. In part, this was due to the setting – an auditorium – which made it hard for students to work in small groups. I wanted to rethink the session. Was there a way to revise the active learning portion to make it more effective?
Description: Working with colleagues, I decided to make a “serious” game for the active learning portion of the session. In this way, I could guide the students’ work, encourage them to work more as a group (they were told that only a limited number of logins was allowed), and give them positive feedback along the way, so even if they got an answer wrong, they would still see the correct one. It was challenging to translate the group activity into a game: one colleague had to learn to use a new program, some activities that worked well in person didn’t work as well in a game, and the game had to be accessible.
Conclusion: I have completed a 2nd year of using this game in class. I’ll discuss the issues remaining: 1) the data are somewhat unclear at this point, because, although the overall average quiz score is higher over the last 2 years with the game, it can be difficult to directly compare results with 2 different teaching methods (2 with and 2 without the game). I am pleased with this year’s score of 76% and that the students have been more animated in class and seem to enjoy the game.
Paper #12: Developing an Interactive Tool to Assist in Complying with the NIH Public Access Policy.
Authors: Merle Rosenzweig, Tyler Nix, Chase Masters; Taubman Health Sciences Library, University of Michigan
Background: On April 7, 2008, the National Institutes of Health instituted the NIH Public Access Policy. In order to apply for and to continue receiving NIH funding, a researcher needs to comply with the Policy. There are many resources that address compliance with the National Institutes of Health Public Access Policy, but most are text heavy. Our goal was to create a tool that would simplify this complex process. We broke down the myriad tasks involved and represented them in an interactive flow chart. By accomplishing each individual task, the NIH Grantee can achieve compliance with the Policy.
Description: The interactive tool was created in two iterations. The first focused on compliance based on the policy’s four available manuscript submission methods. Microsoft Visio was used to develop the simplified PDF process map in which interactive links provided the user with additional information by linking to external documentation. Due to its success, a second, more detailed process map was later developed by the team using the Lucidchart software. It covers the entire submission process and how to track compliance status following submission. Each box on the process map links to a page within the PDF with additional compliance information, and linear navigation is available at the bottom of each page to walk users step-by-step through the process.
Conclusion: Since the National Institutes of Health Health’s Public Access Policy was launched, librarians and informationists have provided support and instruction to those needing to comply with it. Complying with the Policy can be a daunting task that can be time consuming and frustrating. The interactive tool we have developed can help in navigating the complex process required in complying with the Policy. Our talk will outline the software and workflows that the team used to create the interactive process map with the goal of raising awareness for librarians interested in developing similar learning objects to meet their specific training needs.
Paper #13: Doing better with library classes in 2019: Streamlining scheduling, marketing, and data gathering
Authors: Julia C. Stumpff; Hannah J. Craven; Indiana University School of Medicine, Ruth Lilly Medical Library
Objective: Librarians were hesitant to offer recurring classes because attendance was historically low. The scheduling system was cumbersome, and there was no consistent assessment of attendance, class content, or teaching quality. To improve this situation, the library adopted LibCal, marketed classes, and standardized assessment during the spring 2019 semester. The library now has baseline data to make future evidence-based decisions about scheduled classes.
Methods: After surveying current practices and discovering the campus had access to LibCal, two librarians and a staff member quickly trained themselves on and then employed LibCal. LibCal features adopted were: course templates, registration, email reminders, post-test emails with links to assessments, and attendance tracking. The librarians and staff successfully integrated into the marketing team in order to promote the classes across campuses.
Results: With the new system in place, the number of classes offered doubled. This led to three times the number of attendees overall. The average class size increased by one. By providing a hard copy of our assessment survey complimented by an automated email with the online version, the survey response rate was 84%. Class attrition rate was 36%.
Conclusions: Separating the registration and attendance data by month, day, and hour will inform future classes scheduling. Class content and teaching assessment will be analyzed and shared with the class instructors to improve course delivery. In the future, assessment will ask about participants’ locations to determine if distance learners are reached at other campuses across the state. Marketing efforts will continue to evolve.
Paper #14: How are researchers documenting their day-to-day activities?
Honorable Mention Research Paper Award
Authors: Merle Rosenzweig, Marci Brandenburg, Scott Martin, Leena Lalwanim, Mark MacEachern, Yulia Sefvryugina; Taubman Health Sciences Library, University of Michigan
Background: We undertook a pilot project to evaluate the feasibility of launching a full-scale research project to investigate best practices of researchers at our institution for keeping lab notebooks. Our objectives are to learn about practices already present on campus through a series of faculty interviews, and to analyze those practices based on their similarities or differences, their compliance with modern technology, and ease of use and access, as well as to suggest other practices based on our review of tools reported in the literature.
Description: We focused on a narrowly defined group of researches to obtain a more useful set of conclusions upon which to launch a full-scale project. After obtaining IRB approval (HUM00147763), we emailed a biology department at our institution, our sample population, to explaining the project and asking for volunteers. We interviewed six-research faculty from that department about their recordkeeping practices: including print vs. electronic notebook use, data and protocol sharing within the lab, and standard policies. Interviews were recorded and transcribed, and the transcripts were coded and analyzed.
Conclusion: The results provide a snapshot of current disciplinary practice, which will be useful as a benchmark for longitudinal studies of a wider community as campus-supported electronic tools for research documentation are expected to become more widely available in the coming years. They also suggest points of need where the library might productively offer services within the current paradigm to improve access to and preservation of day-to-day research documentation.