By Vijay Ivaturi

At the outset, I would like to acknowledge and thank all those who contributed to development of Pumas. Here is a complete list in no particular order:

Chris Rackauckas; Simon Byrne; Andreas Noack; Patrick Mogensen; José Bayoán Santiago Calderón; Yingbo Ma; Vaibhav Dixit; Xingjian Guo; Shubham Maddhashiya; Harrison Gordin; David Widmann; Joakim Nyberg; Joga Gobburu

I would also like to acknowledge the enthusiastic group of testers who put in time and effort to make this product better and are still working hard. Here is a complete list in no particular order:

Beatriz Gugleiri-Lopez; Joga Gobburu; Elyes Dahmane; Shamir Kalaria; Hechuan Wang; Praneeth Jarugula; Wansu Park; Mathangi Gopalakrishnan; Dinesh Bedathuru

How it all happened

March 2017

Pumas, along with choosing Julia Language to power it, was born out of a hallway discussion with a good friend and a gifted pharmacometrician, Joakim Nyberg during a coffee break at ASCPT conference in March 2017. As a first step, I reached out on Linkedin to Viral Shah, a co-creator of the Julia Language and the co-founder of Julia Computing. Viral, who is now a good friend and and collaborator was excited to learn about my interest to develop healthcare related tools in Julia Language and was immediately on board. I had mentioned to Viral that a lot of our systems require the use of differential equations and he introduced me to Chris Rackauckas, then a PhD student at UC Irvine. Chris blew my mind away with a detailed email, which was in his usual convincing way that I would soon find out about, that had the contents of exactly what I was looking for, and we immediately struck a chord. A small but important summary note:

Chris in his role as the project lead right from the beginning has built Pumas from ground up with a vision that will drive pharmacometrics tooling to new heights. Keep your eyes open.

June 2017

Viral and Chris put together a team to meet at JuliaCon 2017 in Berkeley, CA that could get started on building a prototype. This initial team had another stellar contributor to Julia Language and an amazing statistician, Simon Byrne who was working at Julia Computing. Pumas was originally started under an ambitious open source initiative called Osmosis. For many reasons beyond anyone's control, I decided to move Pumas under CTM's umbrella. It was probably the correct decision given the environment around product development in pharma. Our initial goal was to have a basic pharmacometric modeling engine that could simulate data using compartmental models and estimation using maximum likelihood methods.

Chris and I worked on setting up the simulation aspects while Joakim and Simon chugged away on the estimation methods. Personally, it was a fantastic learning experience.

October 2017

We had a working prototype as early as October and we decided to meet in person at Julia Computing's office in Boston to hash out further details. I was lucky to have had the shortest flight from Baltimore, while Simon and Chris flew in from LA and Joakim all the way from Sweden. The meeting was extremely productive and we came away with a finer blueprint of what we wanted.

The founding team of Pumas in Boston

March 2018

Work slowed down a bit over the holidays but we ramped it up just in time to have a fully working prototype ready by ASCPT 2018, where we met some stakeholders in Orlando to garner and assess interest. Long story short, we quickly realized that we were on our own without any support. However, we were really thankful for the meeting feedback as it helped us define our new directions. Moving ahead without further support would have been a real challenge, but it was right at that point that we found the three biggest supporters and believers in this project - Dr. Joga Gobburu, the CTM Director, Dr Jill Morgan, Chair, Pharmacy Practice and Science, and Dr. Natalie Eddington - Dean of University of Maryland School of Pharmacy. Their moral and financial support re-ignited the project and put us back on track.

Summer 2018

The Google Summer of Code (GSoC) had just begun and most of the Julia Language Community including Chris and Simon were busy mentoring students from around the world on various Julia projects. Chris recommended a few students who worked or were still working with him on the DifferentialEquations.jl ecosystem to join the project. I chatted with a few of them and eventually we had five students who joined the development team in July 2018.

Yingbo Ma and Xingjian Guo worked on the developing and enhancing the capabilities of the differential equation integration for pharmacometric models. They implemented and optimized many stiff ordinary differential equation solvers, and allowed the solvers to make use of automatic differentiation and adjoint sensitivity analysis for use in the estimation algorithms.

Vaibhav Dixit implemented the NLME estimation algorithms (Laplace, FOCE(I), FO) and the diagnostics. He also implemented the first prototypes for Bayesian estimation, along with a thorough investigation on efficient parameter estimation.

Shubham Maddhashiya worked on developing a neat package to facilitate Data Interpolations that form the backbone to handle time varying covariates. He is also the force behind the IVIVC module.

José Bayoán Santiago Calderón handled parsing and creating datasets in the NMTRAN format. He single handedly built the entire Bioequivalence.jl package that performs the simulation and analysis of all major BE designs including highly variable and narrow therapeutic index drugs.

October 2018

Thanks to the expanded team and the support from Joga and Natalie over the summer and early fall, we were feature rich to start putting our work out in the public. This resulted in our first poster at ACoP 2018. The poster received a ton of attention and we knew that we had a major task ahead of us -- to meet everyone's expectations, and our own too!

Harrison Gordin joined us in November 2018 to work on ModelingToolkit.jl, the symbolic backend library that makes it easy to add new features into the Pumas DSL.

We already had a working prototype for Non-Compartmental Analysis (NCA) by ACoP, but it was by no means ready for use. Bill Denney was generous enough by opening up the license for pknca to allow a Julia reimplementation. Yingbo volunteered and developed a Julia version of pknca in just 4 weeks. This was a massive boost to our project.

January 2019

Given the relatively large strides that we had taken, we were confident of putting out an alpha or beta release by late spring or early summer.

David Widmann joined us in early January 2019 to help with integration of Delay Differential Equations into Pumas, the core topic of his Master thesis.

This was the time when we started opening it up internally for testing to get feedback from CTM partners. We wanted to test Pumas's capabilities for Quantitative Systems Pharmacology (QsP) and wanted experts in this field to do this. Vantage Research's Rukmini Kumar and Vikram Prabhakar were kind enough to allow some of their fellows to volunteer their time. One of them, Dinesh, provided significant feedback that allowed really important changes to be made to the core.

March 2019

There was some change of guard happening at the estimation development front as Simon was moving on to a different project. In came Andreas Noack. It did not take long for the team to appreciate what Andreas brought to the table. Trained in statistics, Andreas had already contributed a lot to the JuliaStats world and is a fantastic addition to our team.

The team was in high spirits -- we had conducted extensive testing and were ready to make a public announcement to release Pumas shortly. Joga had received the prestigious Sheiner-Beal award in Pharmacometrics at ASCPT 2019, and revealed during his award speech that we would be releasing Pumas as early as mid-April.

Around the time of ASCPT, we had discussed with CJ Musante the possibility of testing Pumas or rather DifferentialEquations.jl at its core on some of the QsP models that Pfizer was working on in collaboration with Mount Sanai. Chris and Yingbo picked up this challenge and converted a Matlab based ~60-70 ODE model to Julia and did peformance testing. Note that this code in Julia was not optimized at all, and was just a straight translation at best. Julia was ~30x times faster in its un-optimized version comparted to the Matlab equivalent keeping all other factors the same.

This was the sort of verification we needed to tell us that we were on the right path.

May 2019

The team was in frantic preparation for an impending release with minor fixes and continuous testing. Just at this point, we were fortunate to have another outstanding addition to the team, Patrick Mogensen. Patrick is the lead developer of Optim.jl package, which is the backend for our estimation, and along with Andreas will have a major role to play going forward.

As a team we were ready to release Pumas, but just did not anticipate the time it would take to get all the paperwork sorted out to release a product developed at our University.

July 2019

JuliaCon 2019 is happening in Baltimore and we figured there was no better venue to release a Julia based product than here! All the delays and hurdles in releasing it early happened for a reason, and we are finally happy to see the release of Pumas on July 19, 2019.

And now...

This was a combined effort and I cannot imagine pulling off this project without this fantastic team. The end product is always a reflection of the quality of the team and I am proud and feel lucky to have worked with each one of these folks. Kudos also goes out to the whole Julia Community on creating one of the best environments for newcomers and creating a language so elegant while providing massive gains in efficiency in our analysis toolkit.

A special note goes out to Viral Shah and Deepak Vinchii of Julia Computing. They were patient as a saint and provided us a mechanism to work with students that Chris introduced us. It all started with a call, and now we are strategic collaborators on many fronts with a focused goal on providing better solutions for patient care, be it via pharmaceutical drug development innovation or tools for point of care. Thank you.

The journey to this point was thoroughly enjoyable and I hope to continue working with this team to develop products that are useful to the community. Spin some wheels with Pumas and let us know what you think. We are all ears!

Biographies

Chris Rackauckas is a senior research scientist at the University of Maryland, School of Pharmacy, and an applied mathematics instructor at the Massachusetts Institute of Technology. His research focuses on the development of efficient methods for solving pharmacological models, including PK/PD, PBPK, and QsP systems. His background in mathematical systems biology where his PhD work described how stochasticity (randomness) of chemical reactions can be controlled and utilized for patterning in the zebrafish hindbrain. Chris has also engaged in the research of new efficient methods for ordinary, stochastic, and partial differential equations. Along these lines, Chris Rackauckas is the lead developer of the JuliaDiffEq organization for numerical differential equation software which includes the flagship software DifferentialEquations.jl for which he won the Julia Community Prize in 2018.

Simon Byrne is a lead software engineer at Caltech. He has a Ph.D. from the University of Cambridge, 2011. Simon has expertise in management of large and distributed software projects, high-performance computing, statistics and machine learning

Andreas Noack has been a Julia contributor since 2012. His contributions have centered around numerical linear algebra but he has also contributed extensively to Julia’s statistics and parallel computing modules. After finishing his PhD in econometrics, Andreas joined professor Alan Edelman’s JuliaLab at MIT’s Computer Science and Artificial Intelligence Laboratory as a postdoc. During his time at MIT, Andreas collaborated with researchers from Berkeley, UCLA, Harvard, and the Broad Institute on projects ranging from statistical genomics to astronomy and Andreas has given talks on Julia at Stanford and UCLA. In 2017, Andreas joined Julia Computing as the Director of Research.

Patrick Mogensen is an economist with a research background in solving and doing inference of dynamic (discrete) choice models. Patrick has been part of the Julia community since 2015 and has made numerous contributions to the base language, as well as several popular packages, most notably Optim.jl, NLsolve.jl and LsqFit.jl.

José Bayoán Santiago Calderón is a postdoctoral research associate at the Biocomplexity Institute of the University of Virginia, in the Social and Decision Analytics Division. He holds a doctorate degree in economics from Claremont Graduate University. Bayoán has worked as a data scientist and research consultant for several years in the field of residential Demand Side Management (DSM) for utilities, developed and maintained computational economics materials as a lead developer at QuantEcon, and developed pharmaceutical software for the Pumas ecosystem. His research interests include quantitative methods, data science, and machine learning, applied to improving the decision-making of governments. In his spare time, he develops and contributes to open-source software such as packages for the Julia language.

Yingbo Ma was a math major in University of California, Irvine, and he is currently taking a gap year. He is a scientific computing intern in predictive healthcare analytics at Julia Computing, Inc. and the Center for Translation Medicine at the University of Maryland Baltimore. He is very interested in numerical treatments for differential equations and implemented a number of integrators and interfaces in JuliaDiffEq organization. His future goal is to develop new efficient algorithms for solving differential equations and to apply them in real practice.

Vaibhav Dixit is currently a junior undergraduate in Mathematics and Computing at Indian Institute of Technology (B.H.U.), Varanasi. He is mainly interested in understanding the physical world through mathematical models. He has been involved in applications of Statistical and Machine Learning techniques to analysis of dynamical systems modelled through differential equations. He is a contributor to DiffEqParamEstim.jl and DiffEqBayes.jl, both packages for Parameter Estimation and has also worked extensively on DiffEqSensitivity.jl for Sensitivity Analysis, these packages are a major part of the analysis tooling within the JuliaDiffEq suite.

Xingjian Guo was a part-time scientific computing intern in predictive healthcare analytics at Julia Computing, Inc. and the Center for Translation Medicine at the University of Maryland Baltimore. He helped with the development of PuMaS during his Master's studies in scientific computing at New York University. He is interested in the application of high performance computing tools to problems both in academia and in industry, especially those related to differential equations, inverse problems and optimization. He will soon begin working at SoundHound as cross-platform software engineer.

Shubham Maddhashiya is a third year undergraduate student majoring in Ocean Engineering and Naval Architecture at the Indian Institute of Technology, Kharagpur, India. Shubham is interested in artificial intelligence, and especially in its application in the field of robotics. He performs research on path planning and computer vision. Recently, Shubham gained interest in working on various aspects of scientific computing and deep learning. As a member of the Artificial Intelligence team at Kharagpur Robosoccer Student’s Group (KRSSG), he develops autonomous soccer playing robots and participates in international robotics competitions like RoboCup. In Google Summer of Code 2018, Shubham implemented the adaptive multistep method, the adaptive BDF and NDF methods and IMEX multistep methods under the project titled “Native Julia solver for ODEs and DAEs”. At CTM, Shubham is working on developing interpolation techniques that will be used in the larger realm of Clinical Trial Simulation for predictive health analytics.

Harrison Gordin Harrison Grodin is a computer science student at Carnegie Mellon University, with a focus on theoretical computer science and programming language theory. Within the Julia language community, his primary projects are within the domain of computer algebra. He is the author of Rewrite.jl, a principled term rewriting engine, and a core developer of ModelingToolkit.jl, a package for representing and manipulating systems of symbolic equations. He is working as a sceintific computing intern at Center for Translational Medicine and many of his contributions directly impact the Pumas DSL.

David Widmann is a PhD student at the Department of Information Technology and the Centre for Interdisciplinary Mathematics at Uppsala University, Sweden. The main focus of his research is uncertainty-aware deep learning, including the analysis and evaluation of calibration of probabilistic models. David obtained a medical degree and a master's degree in Mathematics from TU Munich. His master's thesis was centered at the numerical analysis of a biological model of delay differential equations and has been leading to many contributions to the JuliaDiffEq organization and in particular the numerical delay differential equation solver DelayDiffEq.jl.

Joakim Nyberg has a Ph.D. in Pharmaceutical Biosciences and M.Sc. in Engineering Physics Experience in system development and project management mainly with high security systems. His research focus is on practical and methodological optimal design. He is the primary developer of the optimal design tool PopED and worked with estimation methods for Pharmacometric models. He experience in therapeutic areas include diabetes, oncology, immunosuppressant and QT-prolongations. He is currently a team leader for a large phase III study, involving project management, QA/QC processes and data security

Joga Gobburu is a world-renowned scientific leader in the area of quantitative disease models and their applications to decisions. He is best known for transforming the field of pharmacometrics into a decision-supporting science. His experience as a senior biomedical research scientist and Director of Pharmacometrics at the Food and Drug Administration (FDA) gives him unique insight into the technical, regulatory, and decision-making aspects in all phases of drug development. He obtained his BPharm and MSc in chemistry from the Birla Institute of Technology and Science, his PhD in pharmaceutical sciences from North Dakota State University, and his MBA from Johns Hopkins University.

Vijay Ivaturi is a clinical pharmacologist and pharmacometrician by training and has worked on multiple drug development and clinical therapeutic programs. His primary research focus is on developing tools and methods to bridge the gap between decision makers and the scientists, with a special focus on clinical therapeutics in pediatrics. Some of the dosing algorithms he has developed are currently being used in pediatric hospitals around the country. Vijay is currently heading a team of scientists involved in democratizing tools and education, the two pillars of progress, for pharmaceutical research. He obtained his doctorate in experimental and clinical pharmacology from the University of Minnesota, which he followed with a post-doctoral fellowship in pharmacometrics at Uppsala University in Sweden.