Time Invariant Trackers of Health Quality-of-Life (Short Form 36) in Eight Year Follow Up After Coronary Heart Surgery

Lindsay, G. M.^1*, Tayyib, N. A.¹, Khedr, S.¹, Tolmie, E. P.², Rideout, A. S.³ & Belcher, P. R.⁴

¹College of Nursing, Umm Al-Qura University, Makkah, Saudi Arabia
²Department of Nursing Studies, University of Glasgow, UK
³Dumfries and Galloway Health Board, UK
⁴Department of Cardiac Surgery, University of Glasgow, UK

*Correspondence to: Dr. Lindsay, G. M., College of Nursing, Umm Al-Qura University, Makkah, Saudi Arabia.

Copyright © 2018 Dr. Lindsay, G. M., et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Introduction

Background
Coronary Heart Disease (CHD) remains the leading cause of death and disability in the industrialized world, having a significant impact on the quality of patients’ day-to-day lives [1]. Because of its impact on health much research and treatments have been pioneered first, in an attempt to prevent this disease, and second in persons with CHD, to improve survival and ameliorate symptoms through medical and surgical means. Coronary Artery Bypass Grafting (CABG) is an accepted treatment showing results in improved longevity [2] for patients with continuing symptoms of CHD after optimal medical management. However, less is known about the wider impact on health quality-of-life (HQoL) and the recurrence of symptoms over time. To investigate the latter perspective an investigation has been undertaken to examine in detail the changes in the HQoL of individuals with CHD [3] who have undergone CABG surgery [4,5,6].

Rather than develop a simpler measure of HQoL from basic principles, we demonstrate how a principal component analysis (PCA) can be used to derive and validate two time-invariant uncorrelated trackers of HQoL based on raw SF36 domain scores calculated from fully completed SF36 patient questionnaires collected prior to CABG, and at one-year and eight-years post-operatively. While the raw domain scores are useful for research purposes, their multiplicity limits their utility in everyday practice for practitioners or patients.

Measuring Health Quality-of-Life and the Impact of Cardiac Symptoms
The health quality-of-life of patients before intervention (Baseline) and at one-year (FU1) and eight-years (FU2) following CABG [4,5,6] was assessed using the health measure Short Form 36 (SF36) developed by Ware et al. [7] together with patients self-rating of the presence and severities of their cardiac symptoms using visual analogue rating scales [8]. The data comprise scores in eight domains of health generated from fully completed responses to the SF36 questionnaire together with the presence and severities of angina and dyspnoea graded on an 8 point scale in unit increments from zero (no symptom) to seven (most severe symptom). The SF36 questionnaire is recognized as a valid and reliable tool for evaluating general health status in a normal population [9] and in the context of patients undergoing percutaneous transluminal angioplasty and changes in symptoms following intervention [10].

The responses to the 36 questions of the SF36 questionnaire were combined [11] to generate scores from zero to one hundred in eight domains of health namely: physical functioning (PF), role limitation due to physical health issues (RP), bodily pain (BP), general health (GH), energy and vitality (EV), social functioning (SF), mental health (MH) and role limitations due to mental health problems (RE). Higher scores in any domain relate to better health status in that domain. Briefly, all SF36 domains showed initial improvement at one-year following CABG, but thereafter mixed behaviour emerged with some SF36 domains showing further improvement while others declined over the time to re-assessment eight-years later [6]. However, by one-year post surgery almost half of the patients reported some level of recurrence of angina and dyspnoea symptoms albeit at a significantly lower level than pre-operatively with a progressive worsening of these symptoms until re-evaluation at eight-years [6].

Changes in the SF36 domain scores across the eight domains of health proved to be a powerful research tool for documenting changes in the HQoL for a population of patients having a health-care intervention and the impact of this intervention over time. However, the eight domains of the SF36 exhibited strong productmoment correlations suggesting the presence of fundamental drivers of HQoL. When we considered how this information could be used by patients and health care providers alike, we realized that the interpretation of the profiles of eight domains of health posed a challenge, particularly when these domains were strongly correlated and exhibited mixed trends. PCA was used to develop simplified, but robust, trackers of changes in HQoL based on the SF36 data collected at different times in the knowledge that PCA is a valid reductionist statistical technique for examining underlying trackers of change in a potentially large set of correlated variables.

A familiar example of PCA analogous to this investigation arises in the analysis of examination marks taken over a spread of subjects. Two significant principal components are manifest, the first being a positively weighted sum of marks measuring overall performance (and scalable to a percentage).

The second is a sum of positively and negatively weighted marks and represents a balance between an individual’s aptitude for STEM and non-STEM disciplines.

PCA has a long established history [12] for understanding trends in large data sets and is recognized for its ability to identify hidden trends in data with a greater fit than statistical correlational techniques alone. The methodology begins with a set of N variables Y₁,Y₂, ... ,Y_N and generates from these N mutually uncorrelated vectors Z₁, Z₂, .... , Z_N which each describe an underlying variation (or trend) in the initial data. These vectors are ordered according to the size of their contribution to the total variance of the initial data [13]. Component Z₁ makes the largest contribution to total variance and describes the most important trend present in the data. Component Z₂ is uncorrelated with Z₁ and accounts for the second most important trend in the data. Component Z₃ is uncorrelated with Z₁ and Z₂ and describes the third most important trend in the data, and so on. The number of components to be used in an application is a subjective decision. When applied to the SF36 domain scores, two principal components were found to describe at least 68% of the total variance at each time of data collection.

Among the many applications of this methodology we mention Berdeaux et al. [14] who used PCA to investigate the efficacy of cataract surgery using a 67 point questionnaires completed pre- and post-surgery. They found that two components described respectively 55% and 6% of variance. Changes in their values were analysed by paired and unpaired t-tests. In a different context Sabharwal and Anjum [15] describe the usefulness of PCA in the context of Big Data applications and list a number of areas in health informatics for which efficient data reduction mechanisms and diagnostic tools could potentially identify problems and the factors causing them. In general, the ability to provide detailed robust evaluation of the effectiveness of a health care treatment is an important and increasingly necessary aspect of modern health-care [16]. There are many perspectives to consider; some of these include the technical competence of the procedure performed, the relief of symptoms of a condition and improvement in overall health status and survival [17]. In addition to these perspectives, differing levels of importance can be attributed to each depending on the stance of the reviewer; to health-care providers, health-care organizations and the recipients of health-care interventions themselves [18].

Brief Review of Previous Work

Sample Composition
Baseline data were collected from 208 patients prior to surgery comprising 45 female and 163 male patients with respective ages at operation ranging from 40 to 79 years (58.01 +/-7.67) and from 40 to 74 years (58.84 +/- 7.57). The difference in the mean ages of male and female patients at the time of operation is 0.83 (p=0.5270). Figure 1 illustrates a histogram of the percentage relative frequencies of the ages of male and female participants.

A chi-squared test applied to the contingency table underlying the histograms of Figure 1 returned χ2=1.8520 (p=0.0308). Thus it is reasonable to assume in this study that male and female patients have similar age distributions at the time of operation thereby allowing male and female data to be pooled for the purpose of analysis. Data were collected at the one-year follow-up after surgery (FU1) from 195 surviving patients [4,5] and at the eight-years follow-up (FU2) from 127 participating patients [6] although 165 patients (79.3%) of the original cohort of 208 patients survived to FU2.

Properties of SF36 Domains
The upper triangle of Table 1 gives the strengths of correlation between pairs of SF36 domain scores at Baseline computed from 201 fully completed SF36 questionnaires, while its reflection about the central diagonal of the Table gives the corresponding p-values of correlations to three significant figures. The central diagonal of the Table contains no useful information and is intentionally left blank to aid clarity.

Table 1. The upper triangle of the table give the strength of correlation between pairs of SF36 domain scores at Baseline based on 201 fully completed patient responses, while its reflection about the central diagonal of the Table gives the corresponding p-values of correlations to three significant figures.

Similar strengths of correlation (not shown) were observed at FU1 and FU2. These observations demonstrate strong correlations between domain scores at each time of data collection despite the fact that responses to the SF36 questionnaire contribute to one domain alone. Previous analyses of this data have focused on properties of individual domains. Panels one to three of Table 2 give the means and standard deviations of the SF36 domains at Baseline, FU1 and FU2. Panel four shows that improvements were recorded in every domain at FU1. Thereafter panel five presents mixed behaviour with mean scores in domains RP and RE showing significant improvements from FU1 to FU2, mean scores in domains SF, MH, EV and BP remaining statistically unchanged, but mean scores in domains PF and GH showing significant deteriorations at FU2, although all scores remained significantly better than their pre-operative values.

Table 2 Panels one to three of the table give the mean domain scores and their respective standard deviations at Baseline, FU1 and FU2. Panels four and five give the changes in mean scores and associated p-values between Baseline and FU1 and between FU1 and FU2.

Properties of Angina and Dyspnoea
Table 3 presents a similar picture for the severity of angina and dyspnoea symptoms at each time of data collection. The results show that both severities are significantly decreased at FU1, but that each has increased from FU1 to FU2, the increase being more pronounced in the case of dyspnoea, but not sufficiently so to return a significant p-value.

Table 3. Panels one to three of the table give the mean severities and their respective standard deviations for angina and dyspnoea symptoms at Baseline, FU1 and FU2. Panels four and five give the changes in these mean scores and their p-values between Baseline and FU1 and between FU1 and FU2.

Table 4 gives the strength of correlations between SF36 domains and the severities of angina and dyspnoea symptoms at Baseline, FU1 and FU2. All p-values at FU1 and FU2 satisfy p<0.001 and are therefore omitted from the Table.

Table 4. Strength of correlation, ρ, between individual SF36 domain scores and the severities of angina and dyspnoea symptoms are given at Baseline, FU1 and FU2. All p-values at FU1 and FU2 satisfy p<0.001 and are omitted from the Table.

Model Construction
The strong correlation structure evident in Table 2 between the eight domains of SF36 at Baseline, and the similarly strong correlation structure existing at FU1 and FU2 (not shown) has motivated the search for reduced trackers of HQoL in the form of expressions combining SF36 domain scores. To be useful, however, the construction of these trackers must be time-invariant so that changes in their values at the different times of data collection reflect changes in HQoL alone uncontaminated by changes in the construction of the tracker. The original study automatically achieves this temporal independence simply by requiring participants to complete the same SF36 questionnaire at each time of data collection. To achieve the same temporal independence for trackers of HQoL when these trackers are developed from combinations of SF36 domain scores at different times involves a two stage approach in which Stage I constructs a model of the trackers while Stage II validates the model.

Stage Ⅰ
The model trackers of HQoL to be developed at Stage I are time-invariant expressions involving weighted combinations of raw SF36 domain scores. Within this context each completed SF36 questionnaire may be viewed as an independent datum irrespective of its time of collection. The model trackers of HQoL are therefore constructed from N=201, N=166 and N=110 fully completed SF36 questionnaires at Baseline, FU1 and FU2 respectively: a total of N=477 independent data.

The data at Baseline, FU1 and FU2 are first de-trended using the mean SF36 scores at each time of data collection. The de-trended data are now pooled and their covariance matrix is calculated. PCA applied to this covariance matrix identifies two principal components which together capture 73.36% of total variance. The first component, abbreviated HS for Health Status, captures 59.23% of total variance and is defined in terms of SF36 domain scores by the expression

Because the value of HS is the sum of positively weighted domain scores, then HS provides an holistic tracker of overall HQoL. Specifically, individuals scoring 100% in each domain of SF36 will likewise record a score of 100% for the value of HS. The second component, abbreviated to HB for Health Balance, is defined in terms of SF36 domain scores by the expression

and captures 14.13% of total variance. Tracker HB is uncorrelated with HS and therefore characterizes a different facet of HQoL. Because the value of HB is constructed by combining positively and negatively weighted domain scores, then HB measures a balance between the physical and mental aspects of HQoL. Above average values of HB reflect a dominance of the physical over mental aspects of HQoL and vice-versa for below average values of HB. The trackers HS and HB are new measures of HQoL. We note in passing that the outcome of PCA for these diseased patients broadly supports the finding of Ware et al. [19] who report that two components accounted for 80% to 85% of the total variance in a general sample drawn from the US population.

Stage Ⅱ
The expressions for HS and HB define a time-invariant model for two uncorrelated trackers of HQoL which together account for approximately 73% of the total variance of the SF36 domain scores. Stage II is concerned with the validation of this model at each time of data collection. The validation comprises phase 1 which seeks to verify that the trackers HS and HB are statistically uncorrelated when applied specifically to data collected at Baseline, FU1 and FU2 followed by phase 2 which seeks to verify that at each time of data collection the trackers together account for percentages of total variance comparable to 73%.

Phase 1 The expressions for HS and HB are used to calculate the values of HS and HB for all patients who returned fully completed SF36 questionnaire at Baseline (N=201), FU1 (N=166) and FU2 (N=110). The correlations, ρ, between HS and HB with corresponding p-values are respectively (0.0398, p=0.5744) at Baseline, (-0.0202, p=0.7958) at FU1 and (-0.0225, p=0.8151) at FU2. Thus HS and HB behave as uncorrelated trackers of HQoL at Baseline, FU1 and FU2 thereby validating the requirements of phase 1.

Phase 2 Values of HS and HB are calculated for each contributing patient and used to compute the variance of HS and HB at Baseline, FU1 and FU2. PCA applied to the 8 x 8 covariance matrices constructed directly from the raw SF36 scores at Baseline (N=201), FU1 (N=166) and FU2 (N=110) determines the true total variance of the raw SF36 domain scores at each of these time points. The percentages of true total variance captured by the variances of HS and HB at Baseline, FU1 and FU2 are subsequently calculated and have respective values 51.93% and 16.45% at Baseline, 67.84% and 11.27% at FU1 and 56.30% and 15.24% at FU2. Thus HS and HB together account for 68.38%, 79.11% and 71.54% respectively of true total variance at Baseline, FU1 and FU2. Thus HS and HB in combination account for at least 68% of total variance at each time of data collection, thereby completing the validation of their use as time-invariant statistically uncorrelated trackers of HQoL.

Results
Table 5 gives the mean values and standard deviations of HS and HB at each time of data collection as calculated from their definitions in section 2.4 using fully completed patient SF36 questionnaires at Baseline, FU1 and FU2. Changes in these trackers between Baseline and FU1, between FU1 and FU2 and between Baseline and FU2 are investigated using paired t-tests. Calculations reveal that the value of HS increases significantly from Baseline to FU1 and thereafter maintains its value indicating that surgery has significantly improved HQoL and that this improvement has been maintained to FU2. Similarly the value of HB decreases significantly between Baseline and FU1 so that surgery has initially preferentially benefited the mental aspects of HQoL. However, there is no significant statistical difference between the values of HB at Baseline and FU2 indicating that surgery has not been long term effective in maintaining the balance between the physical and mental aspects of HQoL.

Table 5 The left hand panel gives the mean (standard deviation) of each tracker at Baseline, FU1 and FU2. The right hand panel uses paired t-tests to investigate changes in these trackers between Baseline and FU1, between FU1 and FU2 and between Baseline and FU2.

Interactions Between Trackers and Cardiac Symptoms
Table 5 shows that HS is always negatively correlated with the severities of angina and dyspnoea symptoms while HB is always positively correlated with these symptoms at each time of data collection. At Baseline, however, neither symptom is significantly correlated with either HS or HB although correlations of symptoms with HB are stronger than those with HS.

At FU1 and FU2 all correlations between HS and HB and the severities of cardiac symptoms are significant. Increased severities of angina and dyspnoea symptoms are associated with decreased values of HS and increased values of HB. The former finding is consistent with expectation and is broadly in line with the behaviour of individual domains at FU1 and FU2. The latter finding is a new result indicating that increased severities of angina and dyspnoea symptoms are particularly detrimental to the mental aspects of HQoL.

Table 6 The Table shows the strengths of correlations, ρ, between HS and HB and the severities of angina and dyspnoea symptoms at Baseline, FU1 and FU2.

Discussion of Results
Literature concerning CABG relating to SF36 health quality-of-life is sparse [4-6]. We set out to generate time-invariant trackers of HQoL based on the eight domains of SF36 as constructed from completed SF36 questionnaires. Although shorter versions of the SF36 questionnaire have been developed, e.g. SF-12, [20], we think that the wider patient enquiry of the SF36 and its sound validation provide a strong basis for further use of SF36 data. PCA makes use of all the available data of the SF36 whereas SF-12 uses only the physical function and mental health domains, and so loses the information content of six domains (and 24 individual questions relating to HQoL).

Baseline Findings
All SF36 domains at Baseline have negative strengths of correlation with the severities of angina and dyspnoea symptoms (Table 4), but only correlations with BP (p-values 0.031, 0.029) and GH (p-values 0.015, 0.037) are statistically significant. Similarly, the severities of angina and dyspnoea symptoms are negatively correlated with HS and positively correlated with HB (Table 6) although no correlations are statistically significant. The positive correlation with HB is a new finding suggesting that increasing severities of angina and dyspnoea symptoms are particularly detrimental to the mental aspects of HQoL.

Baseline to One-Year Follow-Up
Very significant improvements occurred in all domain scores between Baseline and FU1 (Table 2) indicating an immediate benefit of intervention. This behaviour is captured in Table 5 by the very significant increase in HS from its average pre-operative value of 36% to an average value exceeding 55% at FU1. Similarly, Table 5 shows a very significant decrease in the average value of HB post-operatively indicating that surgery has been particularly beneficial to the mental aspects of HQoL. This new finding is not captured by an analysis of individual SF36 domain scores. The results presented in Table 6 also confirm the very significant strengths of correlation between HQoL and the severity of cardiac symptoms at FU1 and confirm the historical results of Table 4.

One-Year to Eight-Years Follow-Up
A mixed message emerged from an analysis of individual SF36 domains with regard to the long term benefit of surgery in the respect that domains RP and RE showed significant improvements, domains SF, MH, EV and BP showed no significant changes, but domains PF and GH showed significant deteriorations (Table 4). By contrast, the trend in HS presented in Table 5 indicates clearly that surgery has been effective in maintaining overall HQoL over this period. However, the value of HB in Table 5 has deteriorated significantly between FU1 and FU2 indicating that surgery has not been effective in maintaining the relative balance of the physical and mental aspects of HQoL. Indeed this balance at FU2 is not significantly different from its pre-operative value. This is a new finding not captured by an analysis of individual SF36 domain scores.

Table 4 indicates strong correlations between improved domain scores and reduced severities of cardiac symptoms consistent with the behaviour of HS in Table 6. Interestingly, the findings in Table 4 have stronger correlations with the relief of dyspnoea symptoms in six out of the eight domains suggesting that patients rate the relief of dyspnoea as more important for HQoL than the relief of angina symptoms. This finding is supported by the results in Table 6 and is consistent with the view that dyspnoea causes more distress to patients because it is less able to be controlled by medication and may be an indicator of a wider deterioration in cardiac function. The strong positive correlations between HB and the severities of angina and dyspnoea symptoms indicate that the relief of cardiac symptoms is more advantageous to the mental aspect of HQoL. This finding is not evident from an analysis of individual SF36 domain scores

The Bigger Picture
In this work PCA has been used as the tool of choice to construct time-invariant trackers of HQoL in respect of its potential ability to achieve greater understanding of the meaning of health. In society today electronic health records have become mainstream in documenting health care and health services utilisation. Opportunities for so called “data mining” using PCA could be endless. Sabharwal and Anjum [15] estimated that the use of Big-Data health analytics to investigate the treasure trove of information in electronic health records, insurance claims, prescription orders, clinical studies, government reports, laboratory results etc. could bring about annual savings of billions of dollars for the US health-care industry. The technique provides the ability to both reduce complexity in large data sets and identify underlying trends or patterns that “tell the story” of, or give voice to, large amorphous data sets.

Conclusion
This investigation has used principal component analysis to develop and validate time-invariant and uncorrelated trackers HS and HB of HQoL based on SF36 data collected from patients who have undergone coronary surgery. Collectively these trackers captured at least 68% of the information content of the raw SF36 domain data at each time of data collection. Patients who underwent CABG have hitherto not been studied by this technique. The tracker HS described overall HQoL and the tracker HB described the balance between the physical and mental aspects of HQoL. Our findings emphasize and extend the versatility of the SF36 questionnaire as a tool for capturing underlying characteristics of HQoL in healthy and diseased populations.

Because these trackers accurately assess holistic aspects of HQoL over time, they are particularly suitable for patient feedback and for assessing the long term effectiveness of surgery. However, the approach used here to construct and validate HS and HB is generic and has the potential to provide trending information on the progression of chronic medical conditions among other applications. In the context of CABG, this approach has clarified that surgery has been long term effective in maintaining overall HQoL, but has been less effective in maintaining a sustained improvement in its mental aspects. Specifically, the analysis revealed the new finding that increasing severities of cardiac symptoms are particularly disadvantageous to the mental aspects of HQoL.

While interest in the use of the SF36 questionnaire has declined over the decade in favor of other forms of evaluation such as customer satisfaction, this article illustrates how the application of other methods of statistical enquiry can provide further insights from data hitherto considered to have been exhausted of information content.

Acknowledgement
The initial phase of this study was undertaken as a research training fellowship funded by the Chief Scientist Office, Scottish Office, Department of Health, Scotland. The cohort follow-up studies were funded by the Chief Scientist Office, Scottish Office, Department of Health, Scotland, reference number 31560, CZH/4/44.

Conflict of Interest
The authors confirm that they have no conflict of interest in any aspect of this study nor in the writing of the submitted manuscript entitled “Time invariant trackers of health quality-of-life (Short Form 36) in Eight Year Follow Up After Coronary Heart Surgery”.

Ethical Statement
Ethical approval was granted by the Local Ethics Research Committee, Glasgow Royal Infirmary in 1996 at the outset of this long term cohort study. Subsequent re-applications for ethics review at 7, 12 and 15 years were approved. The data presented in this paper is anonymous and contains no personal data from the original participants.

Bibliography

1. BHF Living with heart disease.
2. Taggart, D. P. (2013). CABG or stents in coronary artery disease: end of the debate? The Lancet, 381(9867), 605-607.
3. Lindsay, G. M., Smith, L. N., Hanlon, P. & Wheatley, D. J. (2000). Coronary heart disease patients' perception of their health and expectations of benefit following coronary artery bypass grafting. J. Advanced Nursing, 32(6), 1412-1421.
4. Lindsay, G. M., Hanlon, P., Smith, L. N. & Wheatley, D. J. (2001). Assessment of changes in general health status using the short-form 36 questionnaire 1 year following coronary artery bypass grafting. Eur. J. Cardio-thorac Surg., 18(5), 557-564.
5. Belcher, P. R., Gaw, A., Cooper, M., Brown, M., Wheatley, D.J. & Lindsay, G.M. (2002). Are we negating the benefits of coronary artery bypass grafting? Journal of Human Hypertension, 16(10), 691-697.
6. Lindsay, G. M., Tolmie, E., Wheatley, D. J., Ford, I., Hanlon, P. & Belcher, P. R. (2009). Smoking after Coronary artery Bypass; high early follow up mortality. Thoracic and Cardiovascular Surgeon, 57(3), 135-140.
7. Ware, J. E. Jr, Snow, K. K., Kosinski, A. S. & Gandek, B. (1993). SF36: Health Survey Manual and Interpretation Guide. Boston, Mass. Nimrod Press.
8. Lee, K. A. & Kieckhefer, G. M. (1989). Measuring human responses using visual analogue scales. Western Journal of Nursing Research, 11(1), 128-132.
9. Jenkinson, C., Coulter, A. & Wright, L. (1993). Short form 36 (SF36) health survey questionnaire: normative data for adults of working age. Brit. Med. J., 306(6890), 1437-1440.
10. Krumholz, H. M., McHorney, C. A., Clark, L., Levesque, M., Baim, D. S. & Goldman, L. (1996). Changes in health after elective percutaneous coronary revascularization. A comparison of generic and specific measures. Med. Care, 34(8), 754-759.
11. Rand Health: 36-Item Short Form Survey (SF36) Scoring Instructions.
12. Jolliffe, I. T. & Cadima, J. (2016). Principal component analysis: a review and recent developments. Philos Trans A Math Phys Eng Sci., 374(2065), 20150202.
13. Manly, B. F. J. (2005). Multivariate Statistical Methods. Chapman and Hall/ CRC.
14. Berdeaux, G., Viala, M., de Climens, A. R. & Arnould, B. (2008). Patient-reported benefit of ReSTOR® multi-focal intraocular lenses after cataract surgery: Results of Principal Component Analysis on clinical trial data. Health and Quality of Life Outcomes, 6(10).
15. Sabharwal, C. L. & Anjum, B. (2016). Principal Component Analysis as an Integral Part of Data Mining in Health Informatics. Proceedings of 31^st International Society Conference on Computers And Their Applications, CATA pp. 251-256.
16. NICE: Improving health and social care through evidence-based guidance, (2017).
17. McDowell, I. (2006). Measuring Health: A Guide to Rating Scales and Questionnaires. 3^rd ed., OUP, ISBN 978-0195165678.
18. Black, N. (2013). Patient reported outcome measures could help transform healthcare. BMJ (Clinical research ed), 346, f167.
19. Ware, J. E., Kosinski, M., Gandek, B., Aaronson, N. K., Apolone, G., et al. (1998). The Factor Structure of the SF36 Health Survey in 10 Countries: Results from ths IQOLA Project. Journal of Clinical Epidemiology, 51(11), 1159-1168.
20. Rand Health: 12-Item Short Form Survey (SF-12), (2017).