This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Covance Drug Development produces more than 55 million test results via its central laboratory services, requiring the delivery of more than 10 million reports annually to investigators at 35,000 sites in 89 countries. Historically, most of these data were delivered via fax or electronic data transfers in delimited text or SAS transport file format. Here, we present a new web portal that allows secure online delivery of laboratory results, reports, manuals, and training materials, and enables collaboration with investigational sites through alerts, announcements, and communications. By leveraging a three-tier architecture composed of preexisting data warehouses augmented with an application-specific relational database to store configuration data and materialized views for performance optimizations, a RESTful web application programming interface (API), and a browser-based single-page application for user access, the system offers greatly improved capabilities and user experience without requiring any changes to the underlying acquisition systems and data stores. Following a 3-month controlled rollout with 6,500 users at early-adopter sites, the Xcellerate Investigator Portal was deployed to all 240,000 of Covance’s Central Laboratory Services’ existing users, gaining widespread acceptance and pointing to significant benefits in productivity, convenience, and user experience.
Keywords: clinical trial, investigator portal, central laboratory, laboratory results, XcellerateThe operation of a central laboratory, which manages clinical trial testing for a variety of internal and external testing centers, requires a set of specialized systems to ensure rapid and accurate capture and provisioning of results to investigational sites and trial sponsors. These types of systems differ from traditional Laboratory Information Management Systems (LIMS) 1 because the testing requirements and result reporting are primarily protocol- rather than sample-driven. They also differ from traditional Clinical Trial Management Systems (CTMS) 2 in that very detailed tracking of both operational and patient data must be managed and reported to multiple external parties. Covance’s Central Laboratory Services (CLS) 3 business is managed by a collection of information management systems implemented and integrated throughout time. While the labs themselves use LIMS to manage instruments and laboratory workflow, the core study and data management platform is Zavacor, a proprietary transactional system that manages the testing aspects of a clinical trial, including kit ordering, test results, lab reports, and alerts. This legacy system was designed to send lab reports and alerts to investigators via fax. Setting up a new study in Zavacor requires a comprehensive protocol definition that includes visit and test schedules, alert specifications, and information about all participating sites and investigators. To isolate data reporting and transfer processes from the core laboratory operations, data are exported to two data warehouses (one containing protocol information and the other containing laboratory data) that are used for web access and data transfers. Throughout time, a series of applications was developed to meet the needs of various internal and external user communities requiring access to laboratory data. The data warehouses were designed with the ability to contain authorization and access information for each user to ensure proper enforcement of the blinding rules prescribed by the protocol. A significant gap remained, however, in the services provided to investigational sites. A web application previously implemented to provide reporting and document access had substantial limitations in functionality, scalability, and performance.
This legacy application, known as eSiteAccess (ESA), was designed to provide reporting and document access using a generalized design that attempted to simultaneously satisfy the needs of four distinct user roles: external site investigators, external sponsors, internal application support, and internal document publishers. In 2017, we set out to rewrite the investigator-facing modules of the application and create a new portal specifically optimized for a single-investigator user role using modern web architecture concepts and design. A major architectural constraint of this effort was the need to continue to support the legacy applications for the other user roles as well as for investigators using older versions of browsers that do not support modern web technologies. All existing supporting services and database structures also needed to be maintained with minimal enhancements until all legacy applications could be replaced throughout time. Most importantly, the new application needed to provide substantial improvements in usability and reliability for the external sites.
The investigator portal presented here is part of a broader clinical trial management suite, known as Xcellerate, that uses advanced data integration, analytics, and visualization capabilities to improve patient safety, data quality, and protocol compliance throughout the clinical development process and enable greater transparency and oversight of study conduct and performance. 4 –9 Inspired from previous work in discovery, 10 clinical, 11,12 and outcomes research, 13 this solution consists of a number of end-user applications connected to a clinical data repository that supports near-real-time acquisition, mapping, and integration of clinical trial data from any germane source, including laboratory test results, the subject matter of the present work. The Xcellerate Investigator Portal leverages the same architectural and web design principles to provide secure online access to laboratory results, manuals, and training materials through an intuitive interface that enhances user adoption, productivity, and collaboration.
The application design was informed from our in-depth understanding of the current use cases and the anticipated need to incrementally expand the portal’s capabilities throughout time. For the initial release, we identified the following primary use cases for the site investigator role:
Receive, read, and print patient lab reports, including result alerts, lab results, data revision notifications, and test cancellation notifications. This is the most important use case because it is directly related to ensuring patient safety and is also a routine daily activity of investigational sites.
Receive and read lab manuals and study training materials, site communications from Covance or the sponsor, and other general non-study-specific content, including general manuals, training materials, communications, and announcements.
View and search all site lab test results, single-test-result trends among patient visits, single-subject lab test results among all visits, and multiple-test-result trends for a single patient throughout all visits.
Manage personal notification and view preferences. Manage and reset password.An additional key requirement that was identified was the ability for internal staff to emulate a user, allowing support personnel to view the application exactly as an investigator would see it to be able to resolve the most common user questions or issues on first contact.
The application is also required to meet standard clinical trial data security requirements, ensuring that subject information, which includes health information and sensitive test results, is made available only to specifically authorized investigator site personnel. Subject data are de-identified using trial-specific subject identifications, and the system is not subject to HIPAA (Health Insurance Portability and Accountability Act of 1996) compliance requirements.
On the surface, the identified use cases and high-level requirements were similar to those frequently met by commercial off-the-shelf (COTS) document management, content management, and portal frameworks. Our detailed requirements and business logic, however, would require extensive customization to any such COTS platform and force us into a development paradigm that would limit our ability to achieve our product vision. Most commercial frameworks marketed for these purposes are Java-based, which is not our preferred development language. In addition, we had already developed extensive web-based visualization and analytic software assets that we could leverage to quickly bootstrap a custom product to meet our specific needs. Finally, from a strategic perspective, we estimated that a custom, fit-for-purpose solution would be cheaper and easier to maintain, enhance, and support in the long term.
The design of the front end focused on building an easy-to-use, robust, and highly performant interface optimized for the site investigator role (including their supporting staff). We borrowed design patterns and ideas from popular internet and mobile applications that would be familiar to our diverse and global audience. Visualizations are also a key enabling feature in all of the Xcellerate applications; for the Xcellerate Investigator Portal (XIP), the primary focus was on assisting investigators in monitoring patient safety. The key application views are summarized below.
The application landing page uses a familiar card layout concept to provide easy access to all the studies accessible to that individual user ( Fig. 1 ). Besides study identification information, each card also displays the counts of unread documents in the key document categories to facilitate the identification of new information. Simple search, sorting, and favorites features are also provided. The application toolbar, accessible from every page, provides links to general non-study information and also displays notifications for new, unread content.
Xcellerate Investigator Portal (XIP) landing page.
Notification count indicators, similar to those displayed in mobile apps and on social networking sites, are used through the application to notify the user of new, unread content. All user content downloads are logged and may be reported to study sponsors for compliance purposes. Unread content indicators are used on the landing page, the application toolbar, and the individual study pages to assist the user with compliance. When content is required to be read by only a single user at a site for compliance purposes, a status indicator is provided to show if another user from the same trial site has downloaded that content.
As previously mentioned, the primary use cases of the application involve investigator and site staff receiving, reviewing, and possibly printing and/or saving trial test results and result alerts. We use the familiar pattern of a web-based email application inbox, specifically Gmail, as the interface for users to view and access new and historical content ( Fig. 2 ). This is a friendly tabular format with information sorted in reverse chronological order by default and using bold font to indicate unread content. In addition, bulk action features are provided to assist users with repetitive tasks of downloading and printing reports. Simple search and common filters also allow easy retrieval of content. This common inbox pattern is consistently used in all views in which the user receives and accesses document-type content.
Xcellerate Investigator Portal (XIP) lab reporting list view.
Prior to XIP, lab test results could be accessed by investigators only via legacy faxes or as PDF reports. XIP adds online viewing of these data to assist the investigator in identifying trends that may provide additional insights into patient safety. These data can be displayed either at the site level or for an individual subject. Both Excel-like tabular views and visualizations are used to provide convenient and insightful views of all data collected at the site.
The standard test results view ( Fig. 3 , top) displays all of a site’s test results in a user-friendly tabular view. The application’s tabular control provides several options to help find relevant results, including sortable columns, grouping, simple search, visual indicators, unit selection, field-level filtering, saved searches, and export to Excel or CSV (comma-separated values) files. Also provided is the ability to drill down to subject views. In addition to standard test results, some protocols require the presentation of microbiology test results. These data need to be viewed in a unique hierarchical layout, so a separate view was created for the data, as shown at the bottom of Figure 3 .
Xcellerate Investigator Portal (XIP) standard test results (top) and microbiology (bottom) views.
To view test result trends for multiple subjects throughout visits, we added a standard test results trend view, as shown in Figure 4 . This view allows plotting the results of a single lab test for a series of subjects among all visits, normalized in time by starting each patient curve at the initial screening visit. Chart symbols are color-coded by flag value, which usually indicates results that are higher or lower than the normal reference range. Hovering over a subject’s data point displays the specific subject’s reference range, and clicking a data point drills down to the subject’s lab results view.
Xcellerate Investigator Portal (XIP) standard test results trend view.
The subject lab results view ( Fig. 5 , top) provides a way to view all of a subject’s test data in a convenient tabular form organized by visit and test types. Like other test views, many features are provided to help the user find relevant information, including simple search, flagged values, indicators, a flagged values filter, and export to Excel and CSV files. In addition, drill-up capabilities are provided to help compare test results from other patients.
Xcellerate Investigator Portal (XIP) subject lab results (top) and subject test trends (bottom) views.
The subject test trends view ( Fig. 5 , bottom) displays test trends for a single patient’s test results throughout all visits. The data are plotted as aligned small multiple charts, and data points are colored to indicate flagged values, aiding the investigator to view and correlate trends that may help monitor patient safety. Simple search, test group filtering, and a flagged test filter are provided to isolate relevant information.
As mentioned above, XIP leverages the proven web application architecture used for other dashboard and reporting applications in the Xcellerate informatics suite. This is a standard, three-tier architecture composed of a data layer, application layer, and presentation layer ( Fig. 6 ). The data layer consists of the existing relational data warehouses for protocol information and laboratory test result information. A new application-specific database was created to store application-specific configuration data and materialized views for performance optimizations. The application layer consists of a monolithic RESTful web application programming interface (API) that provides endpoints to support reads and updates of the relevant underlying data. In addition, several independent back-end services for file upload, notifications, and user provisioning were retained from the legacy ESA application to provide continuity; the services for report delivery were completely rewritten to allow the system to perform at the required maximum capacity. Finally, the presentation layer consists of a browser-based, single-page application (SPA) that was designed to take advantage of the capabilities of modern browsers and primarily designed for desktop- or large tablet–sized devices.
Xcellerate Investigator Portal (XIP) logical architecture.
The protocol data warehouse is an Oracle relational database 14 composed of multiple physical schemas that separate data by function: security, protocol definition, and report metadata. The laboratory result data warehouse is an IBM Informix 15 relational database. Both warehouses get frequent (mini-batch) updates via ETLs (extract, transform, and load procedures) from the Zavacor system. For the application-specific database, we chose to use the existing Oracle database for convenience and compatibility with the legacy applications and services which were not being replaced. In addition to creating new database tables to support new features in the application, we also implemented a view abstraction layer and materialized views to improve query performance on large datasets.
The RESTful API was built using ASP.NET. 16 A Command Query Responsibility Segregation (CQRS) pattern 17 was used to separate the read and update models with the primary purpose of optimizing read performance in the reporting modules of the application. The CQRS model was built on top of the Dapper simple object mapper for .Net 18 instead of using a “heavy” object relational mapping (ORM) 19 layer. We have found that the transparency and ease of query implementation improve maintainability and allow data engineers to more easily contribute to development efforts.
Other application-layer services were written in .Net as follows:
zReporting Services: Report-processing services that receive report messages in JavaScript Object Notation (JSON) format from Zavacor, convert them to HTML and PDF, save the files to a file share, and insert document metadata into the warehouse. The zReporting services are implemented as a set of .Net Web Services using the Hangfire.IO library 20 to ensure that all required tasks are queued and completed as expected.
Notification Service: A scheduled Windows service that sends out instant notifications and daily digests based on protocol and user preferences.
File Upload Service: A scheduled Windows service that provides semi-automated upload capabilities for lab manuals and training material documents.
Provisioning Services: A set of scheduled Windows services that allow authorized internal users to request access on behalf of site users and have those requests automatically fulfilled without further human intervention for each user provisioning event.
The presentation layer was designed as a JavaScript Single Page Application (SPA) 21 and built using Angular, 22 a TypeScript-based, open-source, front-end web application platform led by the Angular Team at Google and by a community of individuals and corporations. Other third-party open-source and commercial JavaScript libraries were also used: Material Design Lite 23 was used for the base component and layout framework, and ag-Grid 24 was used for all data grid implementations in the application. For visualizations, we used our proprietary D3.js-based 25 library that we originally developed for our Xcellerate Medical Review application. 4 Finally, for identity management and authentication, we used Okta, 26 an OpenID Connect 27 compliant cloud-based identity provider. All connections to the system are strictly required to be secure, incorporating strong Transport Layer Security (TLS) encryption for all traffic to and from the user’s browser.
XIP is used by more than 240,000 investigator staff worldwide on more than 1,800 studies containing greater than 30 million lab report and result alert documents. In addition to study-generated content, all usage and change history are also tracked. With such large amounts of data, performance had to be designed into the system. We used a number of design elements to ensure that the visualizations and document lists load within the performance targets set for the application, including optimized database indexing; materialized views for common queries of slow-changing data; a high-performance micro-ORM data access layer; a CQRS pattern to optimize read queries; horizontally scalable, redundant application servers; a high-performance user interface data grid control; and stateless and asynchronous data flow by default in the application and presentation layers. The system as implemented has been considerably more responsive under load than its predecessor. At peak load, users are able to access the XIP application landing page 50% faster than the comparable ESA page. In addition, lab report access times are up to 80% faster, measuring consistently faster than 2 sec for typical usage scenarios. The fault-tolerant design allows for some system maintenance to be performed while the system remains active for end users, and reduced unplanned outages by more than 60% (comparing XIP availability in 2018 to ESA availability in 2017).
The platform was introduced to Covance’s investigator users in January 2018 by encouraging selected sites to try the new application while retaining access to the existing legacy system, as well as issuing accounts to new users and training them on the new platform first. User feedback was collected via directed surveys, and following positive response, a concerted effort to add more studies and users to the platform began within 3 months of launch. Usage of the new platform exceeded that of the legacy system in September 2018 and continues to grow rapidly ( Fig. 7 ). The new Xcellerate Investigator Portal experiences less downtime, faster response time, and substantially fewer end-user support issues than the legacy system, resulting in improved user satisfaction.
Distinct monthly logins of the new Xcellerate Investigator Portal (XIP) and old eSiteAccess (ESA) legacy applications in 2018.
We have presented the technical underpinnings and user interface of a new web portal that allows secure online delivery of laboratory results, reports, manuals, and training materials to clinical trial sites. The system offers vastly improved capabilities and user experience without requiring any changes to the underlying acquisition systems and data stores. Planned enhancements include the implementation of additional features and enhancements for sponsor users, including a sponsor communication function, as well as online viewing of training materials and web content without the need to download the content locally.
We would like to thank the entire development team who contributed to the design, development, deployment, and support of the Xcellerate platform, as well as our internal and external users for their valuable suggestions and feedback.
Declaration of Conflicting Interests: The authors, all of whom were employed by Covance when the system was built, confirm that there are no known conflicts of interest associated with this publication, and there has been no significant external financial support for this work that could have influenced its outcome. No study sponsor has been involved in the design of the system; the collection, analysis, and interpretation of the data presented herein; the writing of the manuscript; or the decision to submit the manuscript for publication.
Funding: The authors received no financial support for the research, authorship, and/or publication of this article.
ORCID iD: Dimitris K. Agrafiotis https://orcid.org/0000-0002-5604-5692
4. Agrafiotis D. K., Lobanov V. S., Farnum M. A.; et al. Risk-Based Monitoring of Clinical Trials: An Integrative Approach . Clin. Ther . 2018, 40 , 1204–1212. [PubMed] [Google Scholar]
5. McCaffrey R. P., Tevelev B. M., Yang E.; et al. A Statistical Evaluation of Observed Risk Flux in RBM-Managed Clinical Trials . Submitted for publication. [Google Scholar]
6. Farnum M. A., Mohanty L., Ashok M.; et al. A Dimensional Warehouse for Integrating Operational Data from Clinical Trials . Database . 2019, 2019, baz039. [PMC free article] [PubMed] [Google Scholar]
7. Yang E., Scheff J. D., Shen S. C.; et al. A Late Binding, Distributed, NoSQL Warehouse for Integrating Patient Data from Clinical Trials . Database . 2019, 2019, baz032. [PMC free article] [PubMed] [Google Scholar]
8. Farnum M. A., Ashok M., Kowalski D.; et al. A Cross-Source, System-Agnostic Solution for Clinical Data Review . Database . 2019, 2019, baz017. [PMC free article] [PubMed] [Google Scholar]
9. Ciervo J., Shen S. C., Stallcup K.; et al. A New Risk and Issue Management System to Improve Productivity, Quality, and Compliance in Clinical Trials . JAMIA Open . 2019, 2 , 216–221. [PMC free article] [PubMed] [Google Scholar]
10. Agrafiotis D. K., Simson A., Dai H.; et al. Advanced Biological and Chemical Discovery (ABCD): Centralizing Discovery Knowledge in an Inherently Decentralized World . J. Chem. Inf. Mod . 2007, 47 , 1999–2014. [PubMed] [Google Scholar]
11. Farnum M., Lobanov V. S., Brennan M.; et al. Clinical Case: Enhancing Medical Monitoring with Visualization and Analytics . Presented at 2012 IEEE International Conference on Bioinformatics and Biomedicine, Philadelphia, PA, 2012. [Google Scholar]
12. Yang E., O’Donovan C., Phillips J.; et al. Quantifying and Visualizing Site Performance in Clinical Trials . Contemp. Clin. Trials Commun . 2018, 9 , 108–114. [PMC free article] [PubMed] [Google Scholar]
13. Cepeda M. S., Lobanov V. S., Farnum M.; et al. Broadening Access to Electronic Health Care Databases . Nature Rev. Drug Discov . 2010, 9 , 84. [PubMed] [Google Scholar]
