January 10, 2018 – What's the best way to rate the quality and quantity of research produced by orthopaedic surgery residency programs? A new "research impact ranking" provides a more objective approach to assessing and comparing research productivity at US orthopaedic departments, according to a study in the November Journal of Orthopaedic Trauma (JOT), published by Wolters Kluwer.
Partly motivated by the desire to track the success of their own program's research effort, Dr. Louis B. Jones and colleagues of University of Mississippi Medical Center, Jackson, developed an alternative approach to ranking orthopaedic surgery residency training programs in terms of research productivity. The researchers write, "We have created a reproducible and meaningful residency ranking system that will allow programs to evaluate their progress both against their peers and within their own systems over time."
System Ranks Quality as Well as Quantity of Research by Orthopaedic Surgery Trainees
Research, along with clinical care and academic performance, is a key part of the mission of orthopaedic surgery training programs. But research productivity is difficult to measure objectively. Simply counting the number of studies published accounts for the quantity but not necessarily the quality of research produced.
Dr. Jones and colleagues sought to develop a benchmarking system, or "h-index," for residency programs. From a list of 72 orthopaedic surgery journals, they identified 48 "relevant, impactful" journals based on the Impact Factor, a traditional measure of the importance of medical journals; and a newer alternative metric called the Eigenfactor. The journals with the most citations from US residency programs were Journal of Bone and Joint Surgery (American edition), Spine, and Clinical Orthopaedics and Related Research.
Dr. Jones and colleagues then created a database of nearly 103,000 articles published in these journals from 2005 to 2015. These articles were then attributed to the 157 US orthopaedic surgery residency programs. The researchers put their model through several iterations to identify the most effective search process.
The results provided a ranking of the orthopaedic training programs with the most productive research efforts, based on the number of citations in high-quality journals. The article in Journal of Orthopedic Trauma presents the full list of programs and their rankings. The data can also be accessed at the University of Mississippi Medical Center Department of Orthopaedic Surgery and Rehabilitation research page.
Because the ranking system accounted for the number of articles published, larger programs—based on number of trainees and number of faculty—tended to have higher rankings. Each of the six top-ranked programs had more than 1,000 research publications and more than 20,000 citations over the ten-year period studied.
The rankings in terms of number of citations were closely correlated with the number of articles published in the select orthopaedic surgery journals. The rankings were "moderately" correlated with other systems for ranking training programs.
Dr. Jones and colleagues note some limitations of their ranking system—notably including the fact that it's sometimes difficult to attribute articles to a specific residency program.
Nevertheless, they believe their ranking system, based on "clear and reproducible" methods, provides a useful approach to assessing and comparing the research productivity of orthopaedic residency programs. The authors conclude, "It is relevant and based on measurable data that is the cornerstone of an academic surgeon's practice: publishing articles to contribute to the orthopaedic knowledge base."
Click here to read “An Objective Methodology to Assess Meaningful Research Productivity by Orthopaedic Residency Departments: Validation against Widely Distributed Ranking Metrics and Published Surrogates.”
About Journal of Orthopaedic Trauma
Journal of Orthopaedic Trauma (JOT) is devoted exclusively to the diagnosis and management of hard and soft tissue trauma, including injuries to bone, muscle, ligament, and tendons, as well as spinal cord injuries. Under the guidance of a distinguished international board of editors, the journal provides the most current information on diagnostic techniques, new and improved surgical instruments and procedures, surgical implants and prosthetic devices, bioplastics and biometals; and physical therapy and rehabilitation.
About Wolters Kluwer
Wolters Kluwer N.V. (AEX: WKL) is a global leader in information services and solutions for professionals in the health, tax and accounting, risk and compliance, finance and legal sectors. We help our customers make critical decisions every day by providing expert solutions that combine deep domain knowledge with specialized technology and services.
Wolters Kluwer reported 2016 annual revenues of €4.3 billion. The company, headquartered in Alphen aan den Rijn, the Netherlands, serves customers in over 180 countries, maintains operations in over 40 countries and employs 19,000 people worldwide.
Wolters Kluwer Health is a leading global provider of information and point of care solutions for the healthcare industry. For more information about our products and the organization, visit http://www.wolterskluwer.com/, follow @WKHealth or @Wolters_Kluwer on Twitter, like us on Facebook, follow us on LinkedIn, or follow WoltersKluwerComms on YouTube.