Enhancing access to biomedical information through a new search interface for PubMed Central

About NIH

The mission of the Department of Health and Human Services (HHS) is to enhance the health and well-being of all Americans by providing for effective health and human services and by fostering sound, sustained advances in the sciences underlying medicine, public health, and social services.

Within HHS, the National Institutes of Health’s (NIH) National Center for Biotechnology Information (NCBI)—a vital component of the National Library of Medicine (NLM) under NIH—has been a pioneering force in biomedical information since its inception in 1988. Functioning as a premier online biomedical archive, NCBI conducts cutting-edge research, leveraging computational methods to address fundamental molecular-level biomedical challenges and builds production software and databases to support these analyses. With over 6 million daily users, NCBI manages key databases that receive hundreds of millions of monthly visits, including the Sequence Read Archive (which contains more than 25 petabytes of data), PubMed, and PubMed Central. PubMed Central (PMC) is the second most visited U.S. government website, with an average of over 4 million users accessing it daily.

The challenge

The existing PMC search interface was built on an older technology stack that had reached end-of-life. Over time, its infrastructure introduced several limitations:

• Components had become less reliable and increasingly difficult to maintain, which affected performance and made troubleshooting more complex;
• Accumulated technical debt slowed development and made onboarding new team members more challenging;
• The legacy user interface no longer aligned with modern accessibility or usability best practices, including not being mobile-device friendly;
• Inconsistent search behavior, including an autocomplete system that no longer reflected current user search patterns.

NIH needed a fully modernized search system: one that would be cloud-ready, accessible, user-friendly, maintainable for developers, and aligned with the architecture used by PubMed’s search platform.

The approach

Beginning in January 2025, a U.S. Digital Corps Software Engineering Fellow served as the sole front-end engineer designing and building a new search interface for PMC from the ground up.

The Fellow rebuilt PMC Search as a modern cloud-native Django application integrated with multiple backend services, including a new Solr-based search infrastructure, to pursue the following improvements:

• Modernizing the codebase: To ensure long-term maintainability and avoid framework lock-in, the Fellow implemented a Django + VanillaJS + ViteJS approach and refactored major portions of legacy Perl code into Python.
• Redesigning the interface: The Fellow redesigned the entire interface with a mobile-first, user-centered approach. New features included faceted filters, sorting, pagination, a “Results by Year” timeline, persistent user preferences, enhanced accessibility, and completely rewritten autocomplete and filter generation services. The Fellow advocated for and implemented U.S. Web Design System (USWDS) components to ensure compliance, consistency, and reduced maintenance burden. User testing confirmed substantial improvements in clarity, usability, and visual design.
• Improving deployment pipelines: The Fellow brought accessibility to the forefront by adding automated accessibility testing to the development pipeline to ensure Section 508 compliance. Additionally they integrated end-to-end testing to ensure testing was also completed prior to deployment.
• Employing performance engineering: Because PMC serves millions of daily visitors, the Fellow led extensive performance engineering. He optimized front-end performance using Lighthouse, profiled backend behavior with Apache Bench, implemented connection pooling, and tested an experimental asynchronous uWSGI configuration to increase concurrent request handling. To validate real-world scalability, the Fellow used K6-based load testing with datasets representing actual PMC traffic patterns. He tuned CPU and memory settings, pod counts, and autoscaling strategies, and collaborated with the engineer managing the Solr backend to ensure full pipeline performance alignment.
• Improving traffic monitoring: Using Grafana, the Fellow improved site traffic monitoring and observability, providing real-time alerts about spikes in traffic and extended downtime.
• Rebuilding the image conversion pipeline: A vast majority of the articles featured on PubMed include detailed, high resolution images (often .TIFFs) which can be difficult to view, if web-supported at all. .TIFF files must be converted to a web format such as .JPG, .PNG, or, as the Fellow later introduced, .WEBP. Image conversion was previously done by a 20-year old pipeline built using thousands of lines of Perl code that included custom branching logic for specific articles. Over the years, as the context for the branching logic was lost, the image conversion pipeline was left untouched until the Fellow rebuilt the pipeline using Python. This reduced the footprint of the pipeline’s code and introduced the .WEBP file format that could maintain high-quality images in a smaller file size. The Fellow also implemented a testing framework for a QA team to sample and verify the quality of resulting image conversions.

The impact

The modernization of PMC Search dramatically improves access to trusted biomedical information for the public and scientific community. The Fellow completed the work more than two months ahead of schedule, allowing for extended beta testing and reducing the overall project cost by a third.

For the public and research community, the new system provides:

• A faster, more reliable, and more intuitive search experience for the few hundred thousand searches completed per day (search times are down to milliseconds compared to multiple seconds per search);
• Modern features—built using USWDS—that help users locate research findings more efficiently;
• Improved accessibility and mobile usability;
• More accurate autocomplete suggestions, relevant results, and clearer search workflows.
• 40 million images in PMC that are now better served by the new image conversion pipeline.

These improvements streamline the work of scientists, clinicians, students, journalists, and millions of Americans who rely on PMC daily to conduct research or make health-related decisions.

For NIH, the modernization delivers meaningful, long-term institutional benefits:

• Reduced system maintenance costs through modern infrastructure and codebase modernization;
• A dramatically more maintainable and scalable platform aligned with NIH’s cloud strategy;
• A simplified onboarding experience for new developers;
• System reliability improvements that replace fragile legacy components;
• Architectural consistency with PubMed Search, enabling shared best practices and reducing divergence.

The Fellow’s work positions PMC for the next decade of growth while ensuring millions of users can access high-quality biomedical information quickly and reliably.