The scripts in scripts/01_data_prep_real/ turn the record-level data into the tables the model reads: patients, admissions, conditions, costs and lengths of stay by area, condition, age group, sex and IMD quintile, and smoking prevalence for the same groups. They run on the virtual machine, in the order given in chapter 8.
3.1 Hospital episode statistics (HES) definition
The primary data source for acute service utilisation is the Admitted Patient Care (APC) dataset from English Hospital Episode Statistics (HES), for admissions starting between 1 April 2023 and 31 March 2024.
01_process_hes_data.R keeps finished consultant episodes of ordinary elective and emergency admissions, and transfers from another hospital, for patients aged 16 to 109, resident in England, with a recorded provider, sex and IMD decile. Day cases, regular attenders, maternity admissions, births and maternity-related treatment specialties are excluded, as are all episodes of any admission that ended in death (below). Episodes are linked into continuous inpatient spells with hesr::define_spells(), and each spell is one admission. Each patient’s age group, sex and IMD quintile are fixed at their first admission of the year.
Every diagnosis position of every episode is matched against disease_mapping_dictionary.csv on the first three and the first four characters of the ICD-10 code (a four-character match takes precedence). Hip fracture and cervical cancer are counted for women only.
3.1.1 Admissions that end in death
An admission that ends in the patient’s death is not eligible for treatment, as NHS England specifies: the patient is likely to be too ill to be seen by an advisor and has no time after discharge in which to benefit. Such admissions are left out of everything the model counts: admissions, smokers identified, referrals, patients, comorbidities, costs and lengths of stay.
The exclusion is by admission, not by patient. A patient discharged alive in May who dies in hospital in October keeps the May admission, at which they could have been treated, and loses only the October one.
The admissions that end in death are not simply discarded. They go through the same filters as every other admission and are then set aside, for one use only: section 3b counts the patients whose first admission of the year ended in death (hes_index_deaths.csv), so that their deaths are not counted against the patients who survived (chapter 5, “Deaths after discharge”). They never enter the cohort.
3.2 Patient groups
Each admission’s main condition is the first smoking-related condition in diagnosis positions 1 to 5 of its first episode, or “Other” if there is none. Each patient is placed in one group: the main condition of their first admission of the year, with their age group, sex and IMD quintile. All of the patient’s admissions in the year count in that group, whatever each admission was for, so every eligible admission is included and each patient is counted once.
For each group, the pipeline records the number of admissions, the number of patients, the number of admissions per patient (ADMISSION_MULTIPLIERS), the admissions with the group’s own main condition per patient (FOCAL_ADMS_PER_PATIENT, at least 1 because the focal admission is one of them), and the share of admissions with a length of stay of at least one day (pr_los_ge1), which the clinical pathway uses because patients who stay overnight are more likely to be seen by a specialist advisor:
Each patient’s profile of smoking-related conditions is built from all 20 diagnosis positions of all of their episodes in the year. For each group and each smoking-related condition, the comorbidity proportion is the number of the group’s patients with the condition in their profile, divided by the number of patients in the group. Every patient in a group has the group’s main condition in their profile, so the proportion for that condition is 1.
3.4 Admissions per patient for each condition
Each patient’s first admission of the year is their focal admission, the admission at which a patient who smokes is identified and offered treatment. Each admission is assigned to one condition only, its main condition. Two multipliers turn patients into expected admissions:
Focal multiplier (focal_admissions_per_patient): for each group, the number of the group’s admissions with the group’s own main condition, divided by the number of patients in the group. It includes the focal admission, so it is at least one.
Comorbid multiplier (comorbid_admissions_per_patient): for each condition, the number of admissions with that main condition among patients who have the condition but are in another group, divided by the number of those patients. Patients who have the condition but were not admitted for it count as having no such admissions.
The model’s baseline admissions for a condition are the number of patients in a group, multiplied by the proportion with the condition and by the focal multiplier (for the group’s own condition) or the comorbid multiplier (for its other conditions). The admissions for each group’s own condition therefore match the group’s admissions for it, and across all groups the admissions for each condition match the admissions with that main condition in 2023/24. The total admissions in the year are reproduced by the number of patients in each group multiplied by their admissions per patient (ADMISSION_MULTIPLIERS).
3.5 Filling gaps in sparse strata
Stratifying by age, sex, IMD, main condition and comorbid condition leaves some combinations with few or no patients. Where a group has patients but none of them has a condition, the comorbidity proportion is 0. Other missing values are filled by a cascade that takes the average for progressively broader groups: across IMD quintiles, then across age groups, then across the area, then across all areas at the same geographic level. Each step is reported in the console.
Code
# Extract from 01_process_hes_data.Rimpute_explicit_cascade <-function(dt, val_col, group_cols) { dt[, mean_imd :=mean(get(val_col), na.rm =TRUE), by =c(REGION_NAME, group_cols, AGE_GROUP_NAME, SEX_NAME)] dt[, mean_age :=mean(get(val_col), na.rm =TRUE), by =c(REGION_NAME, group_cols, SEX_NAME)] dt[, mean_overall :=mean(get(val_col), na.rm =TRUE), by =c(REGION_NAME, group_cols)] dt[, mean_national :=mean(get(val_col), na.rm =TRUE), by =c(group_cols)]# ... each tier is applied in turn and the number of rows it fills is printed}
The cascade fills comorbidity proportions only for strata with no patients, which have no admissions and so do not contribute to the results. It also fills missing unit costs, lengths of stay, overnight-stay shares and the admissions multipliers.
3.6 Deriving acute spell unit costs
The pipeline calculates the cost and length of each admission. Each spell is matched to the 2023/24 NHS Payment Scheme prices by the core Healthcare Resource Group (HRG) of its first episode. The cost of each spell accounts for:
Admission method: distinct base prices for elective and non-elective admissions.
Length of stay and trim points: the spell’s length of stay is compared with the HRG’s long-stay trim point.
Excess bed days: a per_day_cost is added for every day beyond the trim point.
Short stays: the reduced short-stay emergency adjustment is applied to non-elective stays of less than two days where it applies.
For each smoking-related condition, the unit cost (outcome_unit_cost) is the mean cost of admissions with the condition as their main condition, and the average length of stay (outcome_alos) is the mean length of these admissions, both calculated for each age group, sex and IMD quintile. Means are used rather than medians because the model multiplies them by the number of admissions avoided, and a total is a mean times a count; costs and lengths of stay are skewed, so medians would understate the totals. The average length of stay is used to convert admissions prevented into bed days. The unit costs stay in 2023/24 prices in the prepared inputs; main.R converts them to the model’s price year, 2024/25, when it reads them (chapter 6, “Prices”).
3.7 Admission type of heart disease patients
Section 3c of 01_process_hes_data.R counts the patients in England whose first admission of the year was for ischaemic heart disease, by whether that admission was elective or non-elective (emergencies and transfers from another hospital, as in HES), over all providers (hes_ihd_admission_type.csv). The calibration of smoking prevalence uses the two shares to weight the smoking target for the IHD group (below).
3.8 Geography
The HES pipeline prepares model inputs for three nested geographic levels in one run: England, integrated care boards (ICBs) and NHS trusts, covering the 129 trusts in scope (126 acute trusts and 3 community trusts with inpatient beds). Each admission belongs to the trust that provided it, each trust to one ICB, and England is all admissions to the 129 trusts, so admissions add up across levels. A patient admitted to two trusts is counted as a patient of each, with their full comorbidity profile. The model reads the inputs for the level set by GEO_LEVEL and the area set by ACTIVE_REGION in 00_config.R.
3.9 Smoking prevalence
Smoking prevalence is prepared in three scripts, run after 01_process_hes_data.R, and calibrated each time the model runs. Settings are in section 7c of 00_config.R.
3.9.1 Smoking in the local population (03a_collapse_synthetic_population.R)
The synthetic population (Chapter 2) gives, for each of 46.2 million synthetic adults, the LSOA they live in and the HSE respondent they were drawn from. The script takes only this placement from it. Each synthetic adult is given their respondent’s age group, sex and smoking status, read directly from the HSE 2017 to 2019 files, and their LSOA’s IMD 2019 quintile, from the official IMD file. Smoking status follows the definitions used for England in the hseclean package: cigsta3 (current, ex-regular or never regular cigarette smoker), with smkevr, cigevr, cigreg and cignow where it is missing.
Missing smoking status is imputed with MICE (polytomous regression, 20 imputations) from age, sex, IMD quintile, survey year and the five other characteristics the synthetic population was matched on. Each synthetic copy of a respondent with missing status takes one of the 20 imputations at random, so that every synthetic adult has a single smoking status without giving all copies of a respondent the same draw.
IMD 2019 is published for 2011 LSOAs. A 2021 LSOA formed from a single 2011 LSOA takes its published decile; one formed from several takes the mean of their ranks, cut into quintiles as the official deciles are.
The script then collapses the population to counts of adults by LSOA, IMD quintile, age group, sex and smoking status (lsoa_smoking_counts.csv). Before writing them, it checks that:
every synthetic adult’s respondent is in the HSE files, with the same sex and IMD quintile as recorded in the synthetic population;
the IMD direction is right, by comparing published deciles with ranks and checking two known LSOAs (City of London 001A and Jaywick, Tendring 018A);
people sit in LSOAs in the same IMD quintile as their respondent (99.8% do; the rest are one quintile off, on quintile boundaries, because the synthetic population was built with a different IMD file); and
current smoking rises with the deprivation of the LSOA, as in the HSE (from 10% in the least deprived quintile to 25% in the most deprived).
Any failure stops the script with a description of the problem.
Code
# Extract from 03a_collapse_synthetic_population.R, section 4dx <- pop[, .N, by = .(area =get(IMD_NAME), respondent = donor_imd)]agree <- dx[area == respondent, sum(N)] / dx[, sum(N)]if (agree <0.95) {stop(sprintf("Only %.1f%% of people come from the same IMD quintile as their LSOA, so the local counts can't be trusted.", 100* agree))}
3.9.2 The population each area serves (03b_build_catchment_weights.R)
Each LSOA is weighted by the share of its patients admitted to each trust, from the OHID trust catchment data (Chapter 2). For each MSOA, the share for a trust is the number of its patients admitted to the trust divided by the total number of its patients admitted to any acute trust, and each LSOA takes the shares of its MSOA. ICB and England weights are the sums of the weights of their acute trusts, so the weights nest in the same way as the hospital data. Because the denominator includes trusts outside the 129 in the model, a weight is the fraction of an LSOA’s population that the area serves.
The OHID data cover acute trusts only. Three of the 129 are community trusts with inpatient beds: Norfolk Community Health and Care (RY3), Birmingham Community Healthcare (RYW) and Lincolnshire Community Health Services (RY5). Each takes the catchment of the acute trusts in scope serving the same county (SMOKING_CATCHMENT_PROXY in the config): Norfolk and Norwich, King’s Lynn and James Paget for RY3, University Hospitals Birmingham for RYW, and United Lincolnshire for RY5. Their weight for an LSOA is the shares of those trusts added up. They are left out of the ICB and England sums, because their catchments are copies of acute ones and adding them would count the same LSOAs twice.
The script stops if any OHID trust code is missing from the provider lookup, if a trust in scope has neither patients in the OHID data nor a proxy, if a trust with a proxy has its own catchment in the OHID data, if any MSOA’s counts are inconsistent, or if the MSOAs in the catchment data and the LSOA to MSOA lookup do not match. It lists the OHID trusts outside the 129 every time it runs (qa_ohid_trusts_outside_scope.csv).
3.9.3 Base prevalence for each area (03c_build_smoking_base_grids.R)
For each area at each level, the counts of current, former and never smokers in each age group, sex and IMD quintile are summed over LSOAs, each multiplied by its weight for the area, giving the base prevalence of current and former smoking (smoking_base_grid.csv). This describes the general adult population of the area’s catchment. The base prevalence is the same for every condition: differences between conditions come from the calibration.
Two rules fill cells that the synthetic population cannot support:
a cell with fewer than SMOKING_MIN_CELL_N (30) weighted synthetic adults takes the prevalence for the same cell in the area above: a trust from its ICB, an ICB from England. Every national cell must meet the threshold, or the script stops; and
the 90 and over age group, which is not in the synthetic population, takes the prevalence for ages 75 to 89 in the same area, sex and IMD quintile (SMOKING_AGE_BAND_PROXY). Any other empty age group stops the script.
Each row records which applies in base_source (own, parent or age_proxy).
3.9.4 Calibration to admitted patients (R/calibrate_smoking_prevalence.R, run in main.R)
Smoking is more common among people admitted to hospital than in the general population. At run time, the base prevalence is scaled so that, across all patients in England, modelled smoking matches four sets of targets from user_inputs/smoking_calibration_panel.csv at once:
the overall prevalence (SMOKING_HEADLINE_SOURCE);
the prevalence for each reporting condition group, as a ratio to the overall prevalence, with conditions not related to smoking taking the remainder. For the IHD group the target is the non-elective share of IHD patients times smoking among emergency patients with myocardial infarction (26.3%, Khattak et al. (2024)), plus the elective share times smoking among patients having planned bypass surgery (8.6%, Sewart et al. (2025)), as a ratio to the overall prevalence; and
the shares of patients who smoke in each IMD quintile, sex and age group.
Each target level has one adjustment to the log odds of current smoking, and the adjustments are found together by iterative proportional fitting (raking). On the log odds scale, current smoking in a cell can approach but never exceed the room left by former smoking under SMOKING_PREVALENCE_CEILING (current plus former at most 95%), so every target is met exactly. The targets are shares of patients, because the clinical pathway applies prevalence to patients.
Two things follow from this design. Each condition’s adjustment is the same in every area, age group, sex and IMD quintile, so there is no interaction between condition and the other characteristics: the targets are national ratios by condition, with no information on how they vary by deprivation, age or area. And only current smoking is calibrated. Former smoking stays at the base prevalence, the same for every condition, and never smoking is what is left, so when current smoking is raised for a condition, the increase comes out of never smokers. Former smoking is therefore likely to be understated for conditions such as COPD and lung cancer. This is a stated limitation rather than something the model corrects.
Code
# Extract from main.Rsmoking_calib <-calibrate_smoking_national()smoking_grid <-calibrated_smoking_grid(GEO_LEVEL, smoking_calib)
Both functions are in R/calibrate_smoking_prevalence.R. main.R, build_ch1_outputs.R and build_smoking_inputs_report.R all calibrate through them, so the model and the report always use the same calculation and the same settings.
The adjustments are worked out nationally and the same adjustments are applied to every ICB and trust, so differences between areas come from their populations and case mix, not from the calibration. The calibrated grid and the targets, achieved values and adjustments are written on every run (smoking_prevalence_grid.csv, smoking_calibration_factors.csv), and main.R stops if any patient group is left without a smoking prevalence.
3.9.5 Age limits
Table 3.1: Ages covered by the sources of smoking prevalence.
Source
Ages covered
How it is handled
Hospital data (HES)
16 to 109
Model age groups 16–24 to 90 and over
Synthetic population
16 to 89
90 and over takes the base prevalence for 75–89
Patient level data (calibration targets by IMD, sex and age)
18 and over
The 16–24 target is based on patients aged 18–24
The level of smoking at ages 90 and over is set by the age target in the calibration, so borrowing the 75 to 89 base prevalence affects only how smoking at 90 and over is spread across areas, sexes, IMD quintiles and conditions. If the age target is switched off, patients aged 90 and over would carry the 75 to 89 prevalence, which is higher.
3.9.6 Summaries for the report
scripts/04_reports/main_report/build_smoking_inputs_report.R applies the same calibration to the base grids for England, every ICB and every trust, and saves the prevalence by IMD quintile, sex, age group, condition and area, with the calibration targets, to output/for_report/report_smoking.rds for the report. Patient counts in it are rounded and suppressed in the same way as the other report outputs.
3.10 Referrals to the service (08_process_pld_referrals.R)
The referrals extract (chapter 2) is turned into national referrals by month and calendar year. Each month’s count is broken down four ways (age group, sex, ethnic group and IMD quintile), and each breakdown adds up to the month’s total. Published counts are rounded to the nearest 5, so each is within 2.5 of the true count, and a suppressed count (“*“) is 0 to 4, taken as 2. Each breakdown’s total is an estimate of the same number; the month’s total is the mean of the four, and the script stops if their ranges don’t overlap. The shares of referrals by sex and by IMD quintile leave out”Unknown”. The extract’s quintile 1 is the most deprived, the opposite of the model’s numbering; the calibration tab builder converts it.
The script writes the monthly and annual totals, with the range rounding and suppression allow, and the calibration targets: national referrals for 2023, 2024 and 2025, the full calendar years, and the shares by sex and IMD quintile as checks. All three files stay on the virtual machine.
3.11 Population denominators and mortality rate derivation
The all-cause death rates are deaths per person-year by age group, sex and IMD quintile.
The 02_process_population.R script calculates the population of England aged 16 and over by age group, sex and IMD quintile. Counts by age group and sex are taken from the Office for National Statistics (ONS) Mid-Year Estimates for 2023, and divided between IMD quintiles using the national distribution of the population by IMD within each age group.
The 04_process_mortality.R script divides ONS counts of deaths in England in 2023 by these population counts, giving an all-cause death rate for each age group, sex and IMD quintile.
3.12 Death rates after discharge by condition
Section 3b of 01_process_hes_data.R counts, for England over all providers, by age group, sex and IMD quintile: the patients with each smoking-related condition recorded in the year (hes_condition_patients.csv), and the patients whose first admission of the year ended in death (hes_index_deaths.csv; see “Admissions that end in death” above).
10_process_cause_deaths.R combines these with the ONS deaths by cause and the all-cause rates from 04_process_mortality.R, giving each condition’s deaths a year per patient who has it (condition_excess_mortality.csv) and the death rate from other causes (other_cause_mortality.csv). Chapter 5, “Deaths after discharge”, sets out the method. The same national rates are used for England, each ICB and each Trust, each with its own mix of conditions.
3.13 The lifetime value of each quitter
05_build_lifetime_values.R works out the life years, QALYs and NHS savings of one person who quits for life, by model age band and sex, for the central and the lower assumptions in user_inputs/lifetime_benefits_panel.csv, and writes them to data/base/derived/lifetime_values_per_quitter.csv. It reads Godfrey et al.’s savings by single year of age (data/base/raw/raw_godfrey_lifetime_costs.csv), the ONS life tables (data/base/raw/nlte198020241.xlsx), the NHS Cost Inflation Index and the control panel’s discount rate. It checks that its method reproduces Stapleton and West’s Table 1 and Godfrey et al.’s discounted saving at 35. Chapter 6 sets out the method.