Health Science Journal

Keywords

COVID-19; Efficacy and efficiency; Interrupted time series; Mathematical modelling; Relative risk; SARS-CoV-2

Introduction

Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) causes the corona virus disease (COVID-19). Major features of the disease are failure of multiple organs [1], different symptoms for different age groups [2], selectively targeting adults with co morbidities [3], etc. The infectious virus gets transmitted among human beings primarily via droplets, aerosol and face–oral routes [4] during the asymptomatic and pre-symptomatic period [5] and also when a patient suffering from COVID 19 recovers (convalescence period) by shedding viral RNA [6,7]. SARS-CoV-2 infects the human Angiotensin-converting enzyme (ACE 2) by its receptor-binding domain (RBD) and translate into numerous viral proteins [8]. Further researches are in progress to quantify associations between detectable RNA by real-time reverse transcription–polymerase chain reaction (RT–PCR), a laboratory method for detecting, tracking and studying the COVID-19 virus [9]. To meet the extremely urgent need of a vaccine against the SARS-CoV-2, a new pandemic vaccine development paradigm initiated by compressing the timeline to 1-2 years against the usual development period of 10 to 15 years [10].

Availability of susceptible or resistant hosts can affect the spread of COVID-19 virus in a population. Increase in number of resistant hosts will not allow the virus to spread due to the Herd Effect or Herd Immunity developed after the infection or due to vaccination [11].

Before approval, a vaccine is evaluated in trial stages under optimal conditions on criteria like vaccine efficacy in terms of risk of infection in the group consisting of vaccinated persons (V) vis-a-vis the unvaccinated group (U). In the field conditions, the difference of risk of infection between V and U is a measure of vaccine effectiveness. However, our knowledge of the pathogen of SARS-CoV-2 is not yet exhaustive. Thus, efficacy of a COVID 19 is complex.

Major variables in the field of epidemiology are in nominal scale. Thus, analyses of such nominal-scaled variables in a population are based on frequency measures. The frequencies help us to compute rates of incidence at various time points, ratios between V and U, etc. to quantify Incidence rates (occurrence of new cases), Prevalence rates (presence of the disease), Morbidity rate (percentage of new cases in the population) and Mortality rates (number of deaths per say 0.1 million for specific groups formed with respect to gender or age or by cause). Such measures help to compare and classify groups of individuals in terms of risk and also to find relationships between exposures and disease.

Study of vaccine effectiveness is increasingly important to have better immunization, sustain uptake, in the background of availability of a number of vaccines and different vaccination programmes. The paper aims at exploring different measures of vaccine effectiveness in terms of ratios and also through mathematical and statistical models.

Vaccine efficiency

Efficacy is towards measuring protective effects of vaccination between V and U in controlled clinical trials [12]. But effectiveness of a vaccine, aiming at prevention of the disease is assessed after approval for use in the field conditions. Maintenance of high vaccine coverage is assumed both for vaccine efficacy and vaccine efficiency. Vaccine efficacy (conducted under ideal conditions, say clinical trial) is the biological potency of the vaccine which helps to produce effects and vaccine effectiveness (not under perfectly controlled conditions) is the actual performance of a vaccination programme [13,14].

Factors of vaccine efficacy

In addition to properties of the virus and environmental issues relating to spread of the virus, there are a number of host-associated factors which may have effect on vaccine efficiency. Illustrative list of host-related factors include age, gender, co-morbidities, behaviour (rigorous avoidance of social gatherings), biological (presence or absence of high levels of blood neutrophils and neutrophils extracellular traps (NETs) [15], genetic (variations in human leukocyte antigen (HLA) with SARS-CoV-2 peptides on antigen-presenting cells (APC) etc.), pregnancy (variations in immune-modulatory hormones and adaptive immunity at different stages of pregnancy), commensal microbiota (innate and adaptive immunity), immune imprinting (via infection or via vaccination), etc. [16].

Efficacy endpoints

Efficacy studies of vaccine research need to address adequately the endpoints depending on the types of virus, occurrence and issues relevant to its transmission including etiology, viral shedding, etc. [17]. Dedicated studies are in progress for better understanding of effect of the host factors and biomarkers for immunological protection to the COVID 19 virus and consolidation of various vaccine efficacy findings for different risk groups [18].

Measures

Vaccination program for a population aims towards reducing the (a) rate of transmission with emphasis on the vaccinated group (V) and also (b) trends of occurrence of the disease in the post-vaccination periods. Note that (a) and (b) stated above are highly correlated since occurrence of (b) implies occurrence of (a) and vice versa.

Simple way to assess vaccine efficiency is to compare the positivity rate in percentage i.e. percentage of positive cases among the number of tests done at pre-vaccination and post-vaccination periods. Alternatively, one can compute proportionate reduction in disease attack rate (AR) between U and V groups or find Vaccine Efficacy or relative risk (RR) of disease of persons in the V group by:

                (1)

where ARU and ARV denotes respectively the Attack rate of unvaccinated group (U) and vaccinated group (V).

Note that ,

Thus,     (2)

Popular measures of impact of a vaccine on public health are attributable proportion and effectiveness. The attributable proportion (or attributable risk percent) (AP) is a measure of a causative factor assuming (i) expected risk (mortality rate) for the disease is occurrence of disease in the U-group and (ii) the risk of disease in the V-group exceeds the same for the U-group. AP works well for a single risk factor and is computed as

                (3)

Vaccine efficacy by equation (1) is carried out during a clinical trial. But for the field studies, Vaccine effectiveness (VE) is calculated as

                (4)

where the numerator represents risk difference or excess risk. Risk ratio (or relative risk) (RR) is defined as

                (5)

Clearly, VE=1-RR [19]. 100% RR ⇔ zero incidences in vaccinated persons. Concepts of RR, risk difference and odds ratio are similar and each can be expressed in terms of the others. Thus, association between the exposure and the outcome can be reflected by the simple measure called RR [20].

Alternate ways of computation of RR could be in terms of Bayesian terminology and also from a 2×2 contingency table which are explained below:

                (6)

where D denotes the disease and no disease is denoted by – D; E denotes the exposure and no exposure is denoted by-E.

RR could also be computed as                 (7)

where values (frequencies) of A, B, C and D are obtained from the 2×2 contingency table for a group.

Clearly, higher value of VE implies higher reduction in disease occurrence in the V-group. It is desirable and necessary to have RR < 1 ⇒ reduced risk of the occurrence of the disease in the V-group [21].

The sampling distribution of log(RR) is closed to normal distribution unlike the distribution of RR [22]. The normality of log(RR) helps to find confidence interval and facilitates testing of statistical hypothesis like testing of equality of mean of log (RR) of two different groups i.e.

RR is usually used for randomized controlled trials despite certain advantages of Absolute risk or Risk difference [23].

Rate of transmission of disease in a population of size N can be expressed by where xi is the number of persons affected at time point ti. The quantity depends on the changes in flow of vaccinated individuals (V), unvaccinated individuals (U), susceptible individuals not been infected (S), currently infectious individuals (I), and immune individuals (persons with previous exposure to SARS-CoV-2 either via infection or via vaccination) (R). Thus, can be investigated with solution of suitably formed set of differential equations involving with initial conditions like ; , and along with a set of assumptions. This motivates development of mathematical models to study rate of transmission of disease.

Statistical way to approach rate of transmission of disease is through change in slopes of linear trends obtained from time-series data for the pre-vaccination and post-vaccination periods.

Mathematical modelling regarding spread of COVID-19 has been investigated by several researchers. Discrete-time stochastic epidemic model with binomial distributions was suggested [24]. Strategies for mitigation and transmission for COVID-19 were presented [25]. Mathematical modelling approach to study spread of the COVID-19 was described [26]. Time interval between primary patients (infectors) and the second patients (infectees), known as serial interval was found to be a key parameter for model dynamics and has been used to decide strategies for healthcare programmes [27].

Smith, Rodrigues, Fine [28] distinguished between performance measures based on incidence (cumulative incidence or 'risk') and measures based on incidence density (force of morbidity) and linked these measures with Model 1 and Model 2 for vaccine action assuming proper and effective vaccine administration. Model 2 is applicable only for closed economy (i.e. closed to immigration). The Models also assumed that the vaccination programme results in a proportion 1 − RV of the V-group permanently immune and keeping unchanged the remaining proportion RU. The Model 1 assumes that vaccination changes the hazard (risk per unit time) from a constant to another constant hV = hR hU where hU and hV stand for the hazards for unvaccinated and vaccinated group respectively and Rh is a constant representing the hazard. On the contrary, the Model 2 assumes that vaccination or time do not change the hazard, which is not true for COVID 19 vaccination, where hazard is likely to fall with progress in vaccination. However, both the models fail to consider effect of preventive measures other than the vaccine effects.

Mathematical model of [29] assumed among others, a population of size N with two mutually exclusive groups viz. NU (unvaccinated group) and NV (vaccinated group) where for any time point. This implies a closed homogeneous population with no births, deaths, or migration/immigration. The model solved four differential equations with assumed initial conditions to compute direct, indirect, total and overall effects of vaccine at different vaccination coverage levels where input was the level of reduction in individual transmission rate.

The mathematical model considered by [30] involving a system of non-linear differential equations with transmission parameters enables identification of the transition rates between asymptomatic infected and symptomatic infected individuals. However, despite the suggested models, understanding of transmissibility of the virus is still not clear.

With start of vaccination programme amounting to populationlevel interventions, time series gets interrupted. For evaluation of impact of such population-level interventions from Interrupted time series (ITS) data, there are a number of ITS designs avoiding randomization where data collections are done at a fixed time interval [31,32]. Primary purposes of the ITS studies is to use prevaccination data to estimate correctly the underlying trend and post-vaccination periods along with indication of what would have resulted in the absence of vaccination in terms of two effect measures viz. change in level, showing immediate change after the vaccination and change in slope which quantify the difference in trends in pre-vaccination and post-vaccination programme.

Thus, reduction in trends in pre-vaccination and post-vaccination periods will give a measure of vaccine efficiency. However, presence of autocorrelation (correlation between values of the process at different times) can distort the measures. Presence of autocorrelation can be detected by Durbin-Watson (DW) test and can be accounted for by statistical methods like Ordinary Least square (OLS), Generalised Least square (GLS), Newey-West correction to Standard Errors, Autoregressive integrated moving averages (ARIMA), Restricted Maximum Likelihood (REML), etc. However, DW tests for autocorrelation may perform poorly if the series is not long.

With simulated data, [33] found REML was preferred for longer series, but OLS was preferred for series with ≤ 12 points. For segmented linear regression, [34] used the following linear model with N- time points:

                (8)

where

Yt: Outcome at t-th time point

Dt: An indicator variable representing the post-interruption interval (i.e. where TI represents the time of the interruption.

β0: Intercept in the pre-interruption period

β1: Slope in the pre-interruption period

β2: Change in level

β3: Change in slope

£t: Error term represents deviations from the fitted model

The model assumes amongst others, linearity and additivity between outcome and independent variables; errors are statistically independent (no correlation between consecutive errors for time series data); normal distribution of errors with constant variance (homoscedasticity). However, statistical tests are available to test each of these assumptions. In case of violation of any of these assumptions, the forecasts, trends emerging from the regression model may be distorted. Before using the model, one needs to test for linearity, , detection of autocorrelation with appropriate lag. Goodness of fitting linear trend is reflected by value of R2 (indicates percentage of variance explained) or Adjusted R2 (an unbiased estimate of fraction of the explained variance). If all the above assumptions are satisfied and obtained values of R2 and adjusted R2 are significant, change in slope (β3) can be taken as a measure of impact (efficiency) of the vaccination.

A simpler measure can be obtained by considering slopes of linear trend equations for the pre-vaccination period (b1) and the same for post-vaccination period (b2) It is likely that b1>0 indicating number of cases is increasing with time and b1<0 implying decreasing trend of number of cases with vaccination. Note that time-series data for pre-vaccination and post-vaccination periods can be taken as independent. Test statistic for testing equality of slopes of linear trends obtained from two independent samples of size n1 and n2 i.e. H0: b1=b2 is given by which follows t-distribution with (n1+n2-4) degrees of freedom [35]. In case of non-acceptance of H0 and difference between b1 and b2 will indicate vaccine efficiency. Thus, b1-|b2| could be a new measure of vaccine efficiency which varies with progress in vaccine coverage.

Errors in parameterization of vaccine efficacy may underestimate the impact of mass vaccination in mathematical models to study vaccine efficacy [29]. Model-free measures for evaluation of objectives and models considering administration of vaccine mechanism may be considered independently [13].

Conclusions

Review of model-free measures and outcome measures of mathematical/statistical models towards assessment of efficiency of candidate SARS-CoV-2 vaccine was undertaken for better understanding of the disease dynamics. This is especially important in the context of availability of a number of vaccines and uneven vaccination programmes with different objectives and strategies. Since our knowledge of the pathogen of SARSCoV- 2 is not yet exhaustive, assessment of efficiency of a COVID 19 vaccine is rather complex.

Rate of transmission of COVID-19 and associated mortality rates vary due to a number of factors like age, gender, co-morbidities, genetic & biological factors, innate and adaptive immunity, previous exposure to SARS-CoV-2 either via infection or via vaccination, etc. Thus, measures based on ratios & proportions or output measures emerging from models may be computed for sub-populations defined in terms of the factors of COVID 19. Number of endpoints used in COVID 19 vaccine studies to define efficacy depends on the consequences of infection, and transmission dynamics.

Assumptions involved in various mathematical models like closed homogeneous population may be unrealistic. Each model has limitations too. No mathematical model appears to have considered two transmission paths viz. (i) between unreported symptomatic infected and reported symptomatic infected and (ii) between asymptomatic infected and recovered individuals. In addition, errors in parameterization of vaccine efficiency (VE) in mathematical models may underestimate the impact of mass vaccination. Interrupted time series (ITS) comparing slopes of linear trends in pre-vaccination (b1) and post-vaccine periods (b2) has wide applicability. While spread of virus during pre-vaccination period is reflected by a positive value of with progress in vaccination, b2 is likely to be negative. Thus, the difference of two slopes i.e. b1 −| b2 | could be a new measure of vaccine efficiency. Measures based on ratios and proportions are simple to compute and interpret and facilitate meaningful comparisons. Association between the exposure and the outcome can be reflected by the simple measure called Relative risk (RR). Logarithm of RR log (RR) facilitates calculation of confidence interval since distribution of log(RR) is closed to normal distribution. VE computed from difference of slopes of two linear trend or RR facilitate statistical testing of equality of the vaccine efficiency for two different groups in terms of . Model-free measures for evaluation and models considering administration of vaccine mechanism may be considered independently.

Further research may be undertaken on empirical investigation of various measures of VE and also in areas like heterogeneity of the symptoms associated with SARS-CoV-2 infection, sensitivity analysis of outcomes from models, threshold vaccine coverage to attain a desired level of effectiveness for each measure, etc. Strong international co-ordination and co-operation between governments, regulators, policymakers, and public health bodies will go a long way to ensure effective vaccination programme.

Acknowledgement: Nil

Authors’ Contribution: 100% being the single author

Funding Details: Nil

Conflict of Interests: None reported.

Ethical Statement: The manuscript does not contain clinical studies or patient data.

39139

References

1. Gupta R, Misra A (2020) COVID19 in south Asians/Asian Indians: Heterogeneity of data and implications for path physiology and research. Diabetes Res Clin Pract 165: 108267.
2. Liu Y, Mao B, Liang S, Yang JW, Lu HW, et al. (2020) Association between age and clinical characteristics and outcomes of COVID‐19. Eur Respir J 55: 2001112
3. Guan WJ, Ni ZY, Hu Y, Liang WH, Ou CQ, et al. (2020) Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med 382: 1708-1720.
4. Khan S, Liu J, Xue M (2020) Transmission of SARS-CoV-2, required developments in research and associated public health concerns. Front Med (Lausanne) 7: 310.
5. Oran DP, Topol EJ (2020) Prevalence of asymptomatic SARS-CoV-2 infection: a narrative review. Ann Intern Med 173: 362-367.
6. Zheng S, Fan J, Yu F, Feng B, Lou B, et al. (2020) Viral load dynamics and disease severity in patients infected with SARS-CoV-2 in Zhejiang province, China, January–March 2020: retrospective cohort study. BMJ 369: m1443.
7. To KKW, Tsang OTY, Leung WS, Tam AR, Wu TC, et al. (2020) Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. Lancet Infect Dis 20: 565-574.
8. Jiang S, Hillyer C, Du L (2020) Neutralizing antibodies against SARS‐CoV‐2 and other human corona viruses. Trends Immunol 41: 355-359.
9. Hodgson SH, Mansatta K, Mallett G, Harris V, Emary KRW, et al. (2020) What defines an efficacious COVID-19 vaccine? A review of the challenges assessing the clinical efficacy of vaccines against SARS-CoV-2. Lancet Infect Dis 21: e26-e35.
10. Lurie N, Saville M, Hatchett R, Halton J (2020) Developing Covid-19 vaccines at pandemic speed. N Engl J Med 382: 1969-1973.
11. Joh TJ, Samuel R (2000) Herd immunity and herd effect: New insights and definitions. Eur J Epidemiol 16: 601-606.
12. Haber M, Longini IM Jr, Halloran ME (1991) Measures of the effects of vaccination in a randomly mixing population. Int J Epidemiol 20: 300–310.
13. Greenland S, Freichs R (1988) On Measures and Models for the Effectiveness of Vaccines and Vaccination Programmes. Int J Epidemiol 17: 456-463.
14. Porta M (1987) A Dictionary of Epidemiology. (2nd edn), New York, Oxford University Press.
15. Zuo Y, Zuo M, Yalavarthi S, Gockman K, Madison JA, et al. (2020) Neutrophil extracellular traps and thrombosis in COVID-19. J Thromb Thrombolysis 51: 446-453.
16. Dhakal S, Klein SL (2019) Host Factors Impact Vaccine Efficacy: Implications for Seasonal and Universal Influenza Vaccine Programs. J Virol 93: e00797-19.
17. Lipsitch M, Dean NE (2020) Understanding COVID-19 vaccine efficacy. Science 370: 763e5.
18. Priyanka, Choudhary OP (2021) Vaccine efficacy against COVID-19: A foresight on the host-associated factors. J Formos Med Assoc 120: 1405–1407.
19. Halloran ME, Longini Jr. IM, Struchiner CJ (2009) Design and Analysis of Vaccine Studies, Springer.
20. Sistrom CL, Garvan CW (2004) Proportions, odds, and risk. Radiology 230: 12-19.
21. Carneiro I, Natasha H (2011) Introduction to epidemiology. Maidenhead, Berkshire, Open University Press.
22. Stata Corp LLC. Standard errors, confidence intervals, and significance tests.
23. Noordzij M, van Diepen M, Caskey FC, Jager KJ (2017) Relative risk versus absolute risk: one cannot be interpreted without the other. Nephrol Dial Transplant 32: 1113–1118.
24. He S, Tang S, Rong L (2020) A discrete stochastic model of the COVID-19 outbreak: forecast and control. Math Biosci Eng 17: 13.
25. Tuite AR, Fisman DN, Greer AL (2020) Mathematical modelling of COVID-19 transmission and mitigation strategies in the population of Ontario, Canada. CMAJ 192: E497–505.
26. Cakir Z, Savas HB (2019) A mathematical modelling approach in the spread of the Novel 2019 corona virus SARS-CoV-2 (COVID-19) pandemic. Electron J Gen Med 17: em205.
27. Vink MA, Bootsma MCJ, Wallinga J (2014) Serial intervals of respiratory infectious diseases: a systematic review and analysis. Am J Epidemiol 180: 865-875.
28. Smith PO, Rodrigues LC, Fine PM (1984) Assessment of the protective efficacy of vaccines against common diseases using case-control and cohort studies. Int J Epidemiol 13: 87-93.
29. Shim E, Galvani AP (2012) Distinguishing vaccine efficacy and effectiveness. Vaccine 30: 6700-6705.
30. Khoshnaw SHA, Shahzad M, Ali M, Sultan F (2020) A quantitative and qualitative analysis of the COVID–19 pandemic model. Chaos Solitons Fract 138.
31. Hudson J, Fielding S, Ramsay CR (2019) Methodology and reporting characteristics of studies using interrupted time series design in healthcare. BMC Med Res Methodol 19: 137.
32. Jandoc R, Burden AM, Mamdani M, Lévesque LE, Cadarette SM (2015) Interrupted time series analysis in drug utilization research is increasing: systematic review and recommendations. J Clin Epidemiol 68: 950-956.
33. Turner SL, Forbes AB, Karahalios A, Taljaard M, McKenzie JE (2020) Evaluation of statistical methods used in the analysis of interrupted time series studies: a simulation study. Monash University. Dataset
34. Huitema BE (2011) The Analysis of Covariance and Alternatives: Statistical Methods for Experiments, Quasi‐Experiments, and Single‐Case Studies. (2nd Edn), John Wiley & Sons, Inc.
35. Zaiontz C (2020) Comparing the slopes for two independent samples. Real Statistics Using Excel.