Informing the development of an online resource for patients with oral cancer: triangulation of qualitative data from patients and healthcare professionals
Original Article

Informing the development of an online resource for patients with oral cancer: triangulation of qualitative data from patients and healthcare professionals

Cherith J. Semple1,2^, Rosemary Kelly1^, Ruth Thompson2, Peter Gordon2

1School of Nursing/Institute of Nursing and Health Research, Ulster University, Belfast, UK; 2South Eastern Health and Social Care Trust, Cancer Services, Ulster Hospital, Belfast, UK

Contributions: (I) Conception and design: P Gordon, CJ Semple; (II) Administrative support: R Kelly; (III) Provision of study materials or patients: CJ Semple; (IV) Collection and assembly of data: R Kelly; (V) Data analysis and interpretation: R Kelly, CJ Semple; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

^ORCID: Cherith J. Semple, 0000-0002-4560-7637; Rosemary Kelly, 0000-0001-6709-2227.

Correspondence to: Dr. Cherith J. Semple. Ulster University/South Eastern Health & Social Care Trust, York Street, Belfast, BT15 1ED, UK. Email: c.semple@ulster.ac.uk

Background: The traditional, paternalistic, paradigm of healthcare delivery is no longer acceptable; with transformation placing greater emphasis on shared-care and decision-making, centred on patients’ values, needs and preferences. In this process, patients with cancer often assume a more active role in healthcare. The internet has acted as a likely catalyst and, or facilitator in this process, with proliferation during the global pandemic. Consequently, there has been an expansion of health information being accessed online by patients. A recent scoping review indicated a lack of high-quality evidence-based online resources for head and neck patients, with a paucity of end-user involvement during development. To inform the development and co-design of a high-quality, acceptable online resource for oral cancer patients, aimed at facilitating shared decision-making and treatment preparedness, patients’ and healthcare professionals’ (HCPs) preferences have been elucidated in this study.

Methods: Qualitative research design using semi-structured interviews was employed with patients (n=10) and three focus groups with HCPs (n=21) to understand their perceptions and preference on content, issues to be addressed and key design elements of an online resource to promote decision-making and coping with oral cancer and its treatment effects. Reflexive thematic analysis (TA) was used to analyse both data sets, which were then triangulated.

Results: Three key themes were identified from triangulated TA: (I) key objectives and constructs to underpin the online resource; (II) important content to incorporate within the online resource; and (III) design preferences for the online resource. Participants indicated a preference for online content to be mapped across the key landmarks of the cancer trajectory (at diagnosis, during and after treatment); with tailoring and layering of information; presented through a biopsychosocial lens and incorporating patient experience narratives, to aid contextualising of information.

Conclusions: This research highlights the need to co-produce online resources with key expert stakeholders, integrating factual information alongside patients experience narratives. Incorporating patients’ narrative would appear to be a beneficial source of information to contextualise patient experience, whilst empowering and educating patients to become more proactive in decision-making, self-management and improving health outcomes.

Keywords: Head and neck cancer (HNC); qualitative research; online resource; shared decision-making; patient experience narrative


Received: 15 January 2022; Accepted: 02 June 2022; Published online: 31 October 2022.

doi: 10.21037/fomm-22-1


Introduction

People with head and neck cancer (HNC) often feel overwhelmed by the amount of pre-treatment information they receive from multiple professionals over a short period of time, yet paradoxically, feel unprepared for the changes and consequences from planned treatment (1). Increasingly, patients are looking to the internet for information, exploring it in their own time and at their own pace (2). Additionally, the current global pandemic has illustrated how technology can be used to support patients through virtual clinics, technology apps, and obtaining pre-and post-treatment information online (3-6).

Availability of the internet has enabled patients to access increasing amounts of health information relating to their diagnosis and treatment, which may be helpful in evaluating their options as part of the treatment decision-making process (7). This is concurrently aligned with the recognition that the traditional, paternalistic, paradigm of health care delivery has been reshaped with greater emphasis on shared decision-making; which is fundamentally important in HNC care given the potential for radical changes to physical appearance, eating and speaking following treatment (8). Furthermore, there is a growing body of literature for online intervention efficacy across cancer care (9,10), addressing recommendations within national and international clinical guidelines (11). This spans information provision, dealing with physical and psychological effects of cancer and its treatment, such as fear of reoccurrence (12) and fatigue management (13). Acceptability of web-based interventions is also evolving rapidly, given the known professional barriers to accessing face-to-face consultations, including workforce shortages, and patients’ obstacles, such as ability to take time off work and travel costs (12). Additionally, distressed cancer patients often decline professional one-to-one psychological help (14), with many preferring self-management (15).

Within the field of HNC, there is an emergent recognition from healthcare professionals (HCPs) that the internet can help patients with decision-making, promote self-management and improve quality of life (16), but the quality of available online information is variable (17,18). A recent scoping review published by our team (19) revealed a lack of high-quality evidence-based, web-based interventions for patients with HNC, to promote shared decision-making, preparedness for treatment, or coping after treatment. Of the web-based interventions identified in this scoping review, these mainly focused on treatment-related survivorship issues for HNC patients.

One of the key findings from this scoping review was the paucity of expert stakeholder engagement in the planning and development of HNC online interventions (19). Many web-based interventions were designed by HCPs, then tested with patients, with only a few utilising any form of co-production methodology (1,16). Guided by the updated Medical Research Council (MRC) guidance (20) and the person-based approach (21) for the development of complex interventions, within a complex healthcare system, there is a necessity not only for a robust review of the literature but stakeholder involvement and empirical evidence to enhance intervention development (22,23). Qualitative methods can provide vital insight into the design of an intervention, ensuring it is acceptable and engaging to users who are embedded in the context in which it will be used (21). Furthermore, qualitative research can enhance the development of a theory-driven and evidence-based online intervention, ensuring it is aligned to user priorities (21).

To inform the development of an online resource for patients with oral cancer, qualitative methodology was use in this study, exploring views and preferences of both patients and HCPs involved in delivering direct care to individuals with oral cancer. The focus of this qualitative study was to gain an enhnaced understanding of key informational needs, behavioural and cognitive issues and challenges for patients with oral cancer. Gaining such an understanding will inform intervention development, to promote treatment decision-making and coping with cancer and its treatment effects. Findings from this qualitative study will also inform the design features of an online resource for oral cancer patients, to ensure it is optimally engaging and persuasive, providing insight into the contexts in which patients with oral cancer may engage with this online resource. Insights gained will specify ‘what’ the design objectives must be, and ‘how’ they may be achieved.

The aim of this study was therefore to explore and triangulate patients and HCPs perceptions of appropriate content and acceptable design features for an online resource for oral cancer patients to aid their decision-making, preparedness for oral cancer surgery and promote post-treatment coping.


Methods

Research design

A qualitative research design using one-to-one semi-structured interviews with patients and focus groups with HCPs was employed to enhance the understanding and inform the content, format and delivery of an online resource for patients with oral cancer. Given the range of HCPs involved in the management of patients with oral cancer, focus groups were deemed appropriate as they enable data collection from several people simultaneously, and facilitated in-depth discussion among group members, providing greater insight than individual interviews (24).

Both data sets (patient and HCPs) were initially analysed separately, then triangulated to map out more clearly and fully patients’ needs (25). This process can facilitate confirmatory and complementary information for resource development (26), as it reveals a different aspect of reality. Within the methodological literature, triangulation is a metaphor for strength (27), trustworthiness (28), comprehensiveness (29) and ‘confirmability’ in qualitative studies (28). The study is reported following the COREQ reporting checklist (available at https://fomm.amegroups.com/article/view/10.21037/fomm-22-1/rc) (30).

Sampling and sample

Inclusion and exclusion criteria were established for both participant groups (see Table 1). One local collaborator, a Clinical Nurse Specialist (CNS) in HNC, identified potential participants from one participating healthcare site in the United Kingdom. Participant information sheets (PIS) were shared, and verbal consent obtained for sharing of contact email and, or telephone number with the researcher (RK). The researcher (RK) contacted potential participants to answer questions, obtain written consent and arrange an interview (patients), or participation in a focus group (HCPs).

Table 1

Inclusion/exclusion criteria

Participants Inclusion Exclusion
Patients Aged 18 years Diagnosed with dementia or severe cognitive impairment
Understand written and verbal English Gross psychopathology (defined as “those who are suffering from psychosis, very severe depression or anxiety or other severe mental health conditions”)
Completed treatment for oral cancer
On a curative treatment pathway
Provide written informed consent
Healthcare professionals Member of the HNC team providing direct patient care for people with oral cancer Not a current registrant of a professional healthcare discipline
Holds a professional registration Unable to speak English
Provide written informed consent

HNC, head and neck cancer.

Data collection

The first author (CS) and researcher (RK) have extensive qualitative research experience. Additionally, the research team initially created two topic guides (patient and HCP), informed by the literature and knowledge from subject experts (CS & PG), which were iteratively modified. Individual, semi-structured patient interviews were conducted by researcher (RK) from November 2020 to January 2021 via telephone or video call and were audio-recorded due to COVID-19 restrictions. The researcher (RK) was a Registered Nurse and academic researcher, but not involved in providing care for this patient group.

Simultaneously, the researcher (RK) conducted three focus groups, co-facilitated by first author (CS), using a blend of face-to-face within the healthcare setting (socially distanced according to COVID-19 regulations), or online (Figure 1). These focus groups were audio-recorded and lasted 45–60 minutes.

Figure 1 Audit trail of data collection and analysis. HCP, healthcare professional.

Data analysis

Thematic analysis (TA) is a familiar tool within qualitative research (31). In adopting a relativist approach, we acknowledge that knowledge comes through human interpretation. To understand the human experience of cancer care, the views of those who have experienced that journey are key to informing the development of a resource for future patients.

Each interview recording was transcribed verbatim by the researcher (RK) or professional transcription service. The transcription was checked for validity by two members of the research team (CS, RK) which also ensured familiarity with the data. Data analysis took place guided by Braun and Clarke’s six-step approach (32) to reflexive TA of qualitative data. Initially, the researcher (RK) read and reread the transcripts to understand each participant’s story, to ensure analysis was driven by the data (33). Next, the transcripts were coded manually by the researcher (RK), then codes were developed and collated by the researcher (RK), identifying initial themes. To ensure rigour, credibility and trustworthiness, the codes and initial themes of both data sets were independently analysed by two authors (CS, RK), then triangulated. Through iterative discussion with the research team (RK, CS, PG, RT), the themes for the triangulated data were refined, and final themes were established to ensure the correct meaning of the participants had been captured.

Ethical considerations

This study was conducted in accordance with the Declaration of Helsinki, as revised in 2013 (34), and it was approved by the North West-Greater Manchester Central Research Ethics Committee (No. 19/NW/0720) and research governance from the local participating healthcare trust. Informed consent was taken from all individual participants. Due to the potential sensitive nature of the topic for the patient participant group, a distress protocol was created, ensuring appropriate actions or signposting was provided if required, but utilisation was not necessary within this study. Particular attention was paid to ongoing reflexive practice of the research team to promote transparency and trustworthiness. This included taking field notes during interviews and focus groups, and reflective notes directly afterwards which were regularly discussed with the research team to assess researchers (RK, CS, PG) potential influencing role in the data collection and analysis process.


Results

Ten patients took part in the interviews, 7 males and 3 females. All patients had cancer of the oral cavity, with their primary treatment modality being surgery +/− adjuvant therapies. Their demographics and treatment characteristics are displayed in Table 2. Nine out of ten patients opted for telephone interview, with one video interview conducted.

Table 2

Characteristics and demographics of patient participants

Patient ID Age range (years) Gender Tumour location Treatment modality Time since completion of treatment
1 60–65 F Oral cavity Surgery + radiotherapy 1 year
2 60–65 M Oral cavity Surgery + radiotherapy 4 years
3 60–65 M Oral cavity Surgery + chemoradiotherapy 1 year
4 55–59 M Oral cavity Surgery + radiotherapy Less than 6 months
5 60–65 M Oral cavity Surgery alone Less than 6 months
6 66–70 M Oral cavity Surgery alone 1 year
7 46–50 M Oral cavity Surgery alone 1 year
8 66–70 F Oral cavity Surgery + radiotherapy 2 years
9 36–40 M Oral cavity Surgery + chemoradiotherapy 2 years
10 50–55 F Oral cavity Surgery + radiotherapy 1 year

F, female; M, male.

The HCP focus groups involved 21 HCPs in total recruited from the disciplines within the clinical care team for this patient group (Figure 1). Given this qualitative study is based on the person-based approach (21) to co-design, part of the planning process is engaging with potential users of the online resource to collect their views to inform the content and design. Data saturation is not required, as obtaining in-depth, rich data to inform intervention development rather than theory development, is the purpose of this piece of work. Three key themes were identified from triangulated TA: (I) key objectives and constructs to underpin the online resource; (II) important content to incorporate within the online resource; and (III) design preferences for the online resource. Each theme had a set of subthemes. Illustrative quotes from patient and HCP data sets are displayed in Table 3 to aid transparency of triangulation of data.

Table 3

Themes and sub themes

Theme Illustrative quotes from patient data set Illustrative quotes from HCP data set
THEME I: Key objectives and constructs to underpin the online resource ‘I do feel that the information that I was given, to say that the radiotherapy and the chemotherapy would be a tough road and would be harder on me than the surgery and recovery, I think was very beneficial to know, before I started it’. (Patient 1) ‘Physical preparations, psychological preparation, and one of the big things would be the emphasis on things that they can do to improve their health, to reduce their risks around the time of surgery’. (FG2, Anaesthetist)
THEME II: Important content to be included within online resource ‘Definitely before…Then moving on through your pre-surgery then… a while after surgery and then a year. That is probably, between zero months and twelve months is probably where you need to be targeting, I think’. (Patient 9) ‘It probably would be good to put something on about exercise and building yourself up for your surgery, in that sort of diagnosis phase. Getting as well and as healthy and as fitness strong as possible, for surgery’. (FG1, AHP)
‘I know you have to get all the information and you have to have the worst case scenario because everybody is different and some people have it worse than others, but it’s nice to see somebody talking that went through it to see where you’re going and see what there is you know, there’s life at the end of the tunnel’. (Patient 7) ‘I suppose a lot of patients at that stage (12 months) might not be seeing any AHP’s, maybe they have been discharged or whatever, but I suppose I would be saying that if they are still worried or have concerns, don’t be afraid to lift the phone and contact an AHP or initiate a referral, at least, or have a chat with somebody’. (FG1, AHP)
THEME III: Design features to optimise the online resource ‘I think the idea of a mixture of professional videos and patient videos is brilliant. I think that would be really helpful. I think if there was one almost like a webpage where the journey was listed and patients could go back in after surgery and go right, I have watched to there, now what is going to happen to me. The other thing that might be useful would be after you have had surgery to go back in and look at the ones that were in the hospital and go: did I miss anything?’ (Patient 2) ‘Could you maybe get videos of like, the help with their journey in the hospital? Like, a very short clip of the different teams that will come, say like physio, we are here to help with this and this is the sort of things we might get you to do. Or, a dietician, we are here to… they know what to expect and they are not just getting people coming into the room’. (FG3, Doctor)

HCP, healthcare professional; AHP, allied health professional; FG, focus group.

Key theme I: Key objectives and constructs to underpin the online resource

Triangulated analysis of patients’ and HCPs’ data provided crucial insights into user needs and elicited the most salient constructs that should underpin the online resource. This will consequently help inform the key objectives and design features for the web-based intervention. These included the following items, which will be discussed as subthemes below: (i) informed decision-making; (ii) aid physical and psychological preparedness for oral cancer treatment; (iii) promoting self-management and realistic expectations; (iv) normalising and reducing isolation; (v) resource for family members and (vi) co-design by key stakeholders.

Subtheme (i): inform decision-making

Patients and HCPs both recognised that information on different treatment modalities is instrumental to inform shared decision-making on the most appropriate treatment choice for individuals. This information should include details of both survival outcomes and impact of treatment on quality of life. Both participant groups acknowledged that an inherent challenge is the amount of detail to provide, as individuals’ desire for information will vary, precluding a ‘one-size fits all’ approach. Nonetheless, an over-riding premise was that patients not only wanted information about surgery as a primary treatment modality for oral cancer, but other potential treatment options as a means of informing decision-making:

If you are trying to make a decision about that choice between chemotherapy and radiotherapy and surgery, you need to know what both of those entail. (Patient 2).

Cognisant that the impact of surgery differs depending on the staging and subsite of the cancer within the oral cavity, there was a clear tension, especially for the HCPs, in how an online resource could provide patients with information that was specifically relevant to an individual’s situation without overwhelming them. Although some HCPs and patients felt that the provision of basic and generic information could mitigate against patients becoming overwhelmed, there was a greater preference towards presenting more specific, tailored and detailed information on the type of surgery required, to give patients a sense of the reality of surgery:

If I was a patient, I would probably want some information that was quite specific to the surgery that I am going to have, you know not general information. [Focus Group 1 (FG1), AHP].

In keeping with this premise, there was a careful balance between adequate and patient-centred information, to sensitively aid decision-making but not to provide overly graphical and technical information to create unnecessary fear or alarm. HCPs were especially mindful that elaborate detail may instil unnecessary fear, consequently leading to a negative impact on decision-making ability:

I think you have to be very careful that they don’t lose sight of what is actually important, rather than looking at 100 complications of things that might happen. That will put them off having the operation. (FG2, Anaesthetist).

Patients indicated that on initial receipt of their diagnosis (day of diagnosis) they were laden with concerns surrounding not only the potential treatment that was pending, but the life-threatening nature of their new diagnosis, which had the potential to inhibit their ability to process complex information. Irrespective, the majority of patients reflected on how information not only enabled decision-making but empowered them in the early post-operative days:

To most people I would say that would be horrendous, because you are not going to be able to speak, but whenever you wake up and it hadn’t been done, well that is you upwards, you feel a whole lot better, that is a bonus. (Patient 5).

Professionals also reported the challenges of describing in words alone, the reality of daily life after invasive surgery, and how this could be better addressed through patient narratives or having videos of patients sharing their story. Both patients and HCPs considered that enabling a patient to see a video and hear someone’s story who had similar surgery was especially pertinent. This could enhance and contextualise the information provided by professionals and prove to be an important decision-making tool. Patients’ narratives could present a real-life context to their situation, presenting tangible outcomes on important quality of life issues, and present a patient viewpoint on both coping and functioning after treatment:

Because we try and paint a picture with words of what it is going to be like, but the reality is you can’t. I think videos, pictures, certainly patient experiences, patients telling it, would be really valuable. (FG2, Anaesthetist).

Subtheme (ii): aid physical and psychological preparedness for treatment

Given the gravity of how physically and emotionally demanding major head and neck surgery can be, both participant groups, recognised the need for adequate preparation:

Their time in hospital is as much a psychological battle as it is physical. As much as [the MDT] really make a point of labouring what it is like in hospital because it is really, really grim…[so] patients approaching it with a realistic expectation of how grim the experience is, but methods…ways of sort of coping. (FG2, Anaesthetist).

The findings depicted the need for the online resource to prepare and optimise patients for an overarching view of the whole treatment journey, both physically and psychologically:

I do feel that the information that I was given, to say that the radiotherapy and the chemotherapy would be a tough road and would be harder on me than the surgery and recovery, I think was very beneficial to know, before I started it. (Patient 1).

Physical preparations, psychological preparation, and one of the big things would be the emphasis on things that they can do to improve their health, to reduce their risks around the time of surgery. (FG2, Anaesthetist).

A clear advantage of an online resource would enable patients, with their families to access information and support outside the clinical consultation, at their own pace, promoting preparedness for treatment:

But I also think that even for those people to go home and to consolidate that as if an aide memoire type of thing. (Patient 3).

Patients also considered that an online resource should help map the patient journey and provide detail on what was ahead. For patients having oral surgery, both participant groups considered the importance of preparedness for functional changes, especially for altered speech and swallowing issues:

They tend to think, yes, I will be eating and drinking within a few days…um…no, you might be having a few sips of water or you might be on a teaspoon of custard, but it could be weeks and weeks before you are eating enough to even maintain nutrition. (FG1, AHP).

Subtheme (iii): promote self-management (empower, motivate and goal setting before and during recovery)

Before surgery, HCPs alone identified the importance of a prehabilitation section promoting smoking cessation, alcohol reduction, good nutrition and physical exercise for patients. This could offer patients some control, with proactive engagement in optimising their health, while surgery is awaited:

It probably would be good to put something on about exercise and building yourself up for your surgery, in that sort of diagnosis phase, and maybe having something about that. Getting as well and as healthy and as fitness strong as possible, for surgery. (FG1, AHP).

Whereas, both patients and HCPs highlighted the importance of goal setting, being pre-emptive and motivated in the initial recovery phase to minimise complications and promote recovery, albeit challenging following major HNC surgery:

See I pushed myself cos I wanted to get out for Christmas but she [nurse] said ‘there are people that get into that bed and just lie there, expecting to get better. And it won’t happen. A wee detail, that part of the recovery is up to themselves’. They [MDT] can only do so much, then you have to help yourself… Because you don’t feel like it. There’s many a time I was lying in the bed, and they said go for a walk and you felt so battered just going for a small walk and it was only some steps down the corridor but the surgeon was always very adamant about that. (Patient 7).

Findings denoted that oral cancer treatment often has lasting and late effects; necessitating ongoing rehabilitation interventions to optimise post-treatment function. With this in mind, participants from both groups recognised that the online resource should empower patients to engage in self-management with exercises to optimise mouth opening, neck and shoulder movement and lymphoedema. It should also encompass signposting information to help meet specific post-treatment needs such as return to work issues, or emotional support:

Some of the things they will have to do will be lifelong exercises even. You know from a scar tissue point of view and their jaw opening. They still need to work on that every day, you just don’t get your range and that is you. You would still need to be doing those exercises regularly, or some things, the scar tissue, especially post radiotherapy (FG1, AHP).

Subtheme (iv): normalise and reduce isolation

Patient data demonstrated that individual’s perceptions of themselves can alter or diminish after treatment for oral cancer. The online resource should provide information to help patients prepare emotionally for such ‘losses’, whether these ‘losses’ are short or long-term and help individual’s recognise the importance of adjusting to a ‘new normal’. Having an understanding and some insight from other patient’s lived experience could also reduce the isolation ensued from a cancer that is relatively uncommon, with little opportunity to meet others who have confronted a similar situation:

Just before the diagnosis and during the whole procedure and certainly after it for months, maybe 18 months after it, I felt that I had missed out on communicating with people. In other words, sharing or listening to. (Patient 3).

It was also recognised by HCPs that this resource might be helpful for patients in the recovery phase of their cancer trajectory, by presenting an understanding what ‘normal’ might look like, so they could benchmark and normalise this period:

You know trying to find a bit of yourself again, although that might be a new normal, but just trying to get enjoyment out of that new normal and those sort of things as they move forward. (FG1, AHP).

Subtheme (v): resource for family members

An issue of key importance to patients was the availability of the online resource for family members or carers. Offering practical advice for family members or carers was considered instrumental, as family members were highlighted as pivotal in providing necessary care and support following hospital discharge:

Another aspect might be families as well of those associated and those related or close to patients. In terms of how do you cope with a person who has just come through that trauma at home? And practical things like eating and speech and so on. (Patient 3).

Some patients considered photographs would be helpful for families to gain insight and understanding of what patients will look like postoperatively, especially in the initial recovery period:

I think a photograph of somebody, so that they know, when eventually I do get up to visit, his is what it might look like. Particularly then if they have got, maybe not small children but teenagers might want to come. To have a photograph, this is what your mum or dad looks like, I think would be useful. (FG1, AHP).

As alluded to earlier, too much detailed pictorial information of the initial post-operative period, might have the converse effect and prove shocking for people at such a sensitive time:

I had lots of tubes, I didn’t look at myself particularly, because I didn’t really care. It is finding a balance between frightening people. The thought of the trachy was scary enough for me. … I suppose for some people it might be useful to see it. And yes, maybe for family members. (Patient 10).

HCPs felt the online resource could be helpful for family members who are unable to attend appointments, which was reported as a prevalent issue during the COVID-19 pandemic. Furthermore, it could be used by patients, in conjunction with family members outside of patient-clinician consultations, as a means of recapping on evidence-based, relevant information about their diagnosis and pending treatment:

I think families will use the resource. I was thinking that if families can’t come up to the appointments, you know maybe you have just got, if it’s a husband you have got a wife who comes, who hears all the information. But if you have got daughters and sons who are also wanting to know what is going on, they are the ones I think as well, will be on that website. (FG1, AHP).

Additionally, HCPs in one of the focus groups identified how the resource might be used as a training tool for staff to educate them about patient experiences:

I think it would be great with the juniors coming through, to learn about the patient journey, because from that, I don’t know if you would be happy for the juniors, whenever they start, to access that. Because these juniors don’t know anything about head and neck cases. (FG3, Doctor).

Subtheme (vi): co-design by key stakeholders

Participant groups were resolute that both patients and HCPs should be involved in planning the layout and content of the online resource to enhance the acceptability and effectiveness of the intervention:

‘one of the things that I believe strongly is that the patients should be part of the process’. (Patient 2).

There was some concern expressed by a few professionals that patients may not wish to be involved in providing video narratives for the resource, with each patient’s journey being very personal to them, and not wishing to share things at a particularly sensitive time, or remember their experience at a later date, but one surgeon pointed out that:

I said, when you came to see me on the first day, if I had of asked you, they said, I would have been scared but I would have been happy to be involved in that and I would have loved to have had that resource to actually sit and look at somebody. (FG3, Doctor).

Key Theme II: Important content to be included within the online resource

Both patients and HCPs were in overwhelming agreement that the online resource should map across the key pillars of the patient trajectory, namely before surgery, during active treatment and post-treatment, especially the first 12-months:

Definitely before, obviously the diagnosis part of it, I think was the hardest for us, well probably for anyone really to come to terms with I suppose. Then moving on through your pre-surgery then… a while after surgery and then a year. That is probably, between zero months and twelve months is probably where you need to be targeting, I think. (Patient 9).

Patients were particularly keen to have access to patient stories that would be both prospective and longitudinal, so insights could be gained at pivotal points across the cancer trajectory, enabling progress and recovery to be mapped:

I think if there was one almost like a webpage where the journey was listed and patients could go back in after surgery and go right, I have watched to there, now what is going to happen to me. The other thing that might be useful would be after you have had surgery to go back in and look at the ones that were in the hospital and go: did I miss anything? (Patient 2).

It’s nice to see somebody talking that went through it to see where you’re going and see what there is you know, there’s life at the end of the tunnel. (Patient 7)

Patients considered at diagnosis (pre-treatment) the content should focus on imaging (CT, MRI and PET-CT scans), role and function of the multidisciplinary team, hospital environment during the inpatient stay, whereas professionals considered the importance of promoting prehabilitation. During treatment both participant groups considered there should be an overview (not exquisite detail) on effects of surgery, necessary tubes, drips and drains and who will be involved in their immediate post-operative care:

Like, a very short clip of the different teams that will come, say like physio, we are here to help with this and this is the sort of things we might get you to do. Or, a dietician, we are here to…they know what to expect and they are not just getting people coming into the room. (FG3, Doctor).

There was a collective consensus that there should be detailed information covering a range of applicable and important survivorship issues, spanning emotional well-being, eating and drinking, speech, appearance and fatigue. There should also be signposting to other therapeutic support services.

I suppose a lot of patients at that stage (12 months) might not be seeing any AHP’s, maybe they have been discharged or whatever, but I suppose I would be saying that if they are still worried or have concerns, don’t be afraid to lift the phone and contact an AHP or initiate a referral, at least, or have a chat with somebody. (FG1, AHP).

Key Theme III: Design features to optimise the online resource

Clearly both participant populations purported and highlighted the importance of tailoring information to meet the needs of individuals, which can be challenging with an open-access online intervention.

The internet refuses nothing and people can go on, but the purpose of this project is that it is to benefit the patient as much as we can with this bespoke information that will shape their outcome and influence their decision-making process towards this surgery and this treatment. (FG3, Doctor).

To overcome the barrier of patients receiving an overwhelming amount of detailed information that is not applicable to their situation, [i.e., pursual of a patient narrative following a subtotal glossectomy and free tissue reconstruction, when the requirement is for a partial glossectomy alone], such bespoke components with specific patient stories could have restricted access and be password protected.

Each individual video could have a code, or each individual section could have a code. You could then send the codes that are relevant to that patient. So, then they would never open that video that is not going to apply to them, because they haven’t been given the code to open it. (Patient 2).

Patient and HCPs considered that a multimedia approach would be advantageous, to include a combination of advice and fact sheets, patient narrative videos and professional videos to enhance the sharing of pivotal information at diagnosis, during and after treatment. Other features identified by HCPs were ease of navigation, with clear navigations tools displayed to avoid important information being missed or poor engagement with the online resource. Furthermore, HCPs indicated that the readability of the text incorporated within the online intervention should be checked against readability scores for the general population, given the preponderance of patients with oral cancer from poor socioeconomic backgrounds.


Discussion

This paper uniquely contributes triangulated findings from patients and HCPs, highlighting salient design features for an online resource for oral cancer patients, including what content should be provided, across what timepoints, to aid their decision-making, promote preparedness for treatment and enhance post-treatment coping. Key findings indicated this online resource should map across three key landmarks in the cancer journey; namely, before surgery, during treatment and post-treatment. Patients had a desire for the online resource to provide an awareness of different treatment modalities available for oral cancer (35), sequelae related to their particular HNC sub-type (36,37), treatment-related consequences from a patient experience perspective (38) and anticipated long-term recovery process (39). Patients and professionals in this study highlighted that end-users understanding of the treatment trajectory should be optimised through tailoring and layering of information, to avoid patients being overwhelmed with inappropriate information and potentially misguiding decision-making (40).

Participants in the current study identified a pivotal role to embed patient experience narratives within the online resource, to contextualise information and enhancing understanding of the short and long-term lived effects of treatment, which could aid shared decision-making and preparedness for treatment. Checklin et al. [2020] (41) also purported that hearing from other patients about their experiences at key time-points in the journey can enhance real-life understanding. Patients also recognised the value of being able to go back and review patient narrative videos at different time points, to confirm their experience, or to flag up questions for their multidisciplinary team members. Hence, offering a benchmark to manage uncertainties (42) and gain insights how to cope with everyday problems (43). Furthermore, knowing that others are tackling similar problems can reduce feelings of isolation, and reassurance that one’s experiences and reactions are ‘normal’ (44), offering a mechanism for support.

In addition, it would appear that acquisition of online information has the potential to positively enhance patient-clinician relationships (45), improve health outcomes by increased treatment compliance (46), improve self-management and can promote engagement of family members (47). This may be of particular importance for patients from lower education backgrounds, as they are less likely to play an active role in shared decision-making (45). Furthermore, it is important to note that there are persistent disparities in computer literacy and internet access amongst patients from lower socioeconomic settings (48).

The profusion of online patient experience information has dramatically influenced the provision of health information, equipped people to make better health-care choices, improved health literacy and understanding of illness and enhanced access to more supportive services (49). Nonetheless, it is important to denote the potential limitation of such information. Firstly, face-to face or peer group meetings are favoured by patients as a way of understanding the realities of the HNC experience, but this is not always possible, particularly in light of the current pandemic (50). Additionally, not all patients feel comfortable talking to others about their condition (38), but more importantly, patient experience information can have an unintended negative impact. Open-access, online patient narratives may raise anxiety, with limited access for patients to gain timely clarity from their healthcare team on how representative the information is for their unique situation. Furthermore, if patient narratives feature only a small number of unrepresentative patients’ stories, this can distort and misrepresent patient reality, negatively influencing decisions (49).

This study identified that an online resource should offer information and support through a biopsychosocial lens. The biopsychosocial impact of oral cancer has previously been recognised (11,36,37,51), with evidence that being unprepared for some of the side-effects or challenges during and following treatment can exacerbate feelings of depression and anxiety (52). This is particularly evident in relation to key survivorship areas, such as social eating and drinking (36), speech (53) and intimacy (54). Developers of online resources, whose aim is to promote positive coping for HNC patients, should ensure content is presented within a biopsychosocial context, to be maximally effective and relevant (37,55).

Several of the participants in the current study identified important design features, such as ease of navigation and use of multimedia within the online resource. Within the sphere of digital intervention development there is recognition that these are key elements of web design that need to be harnessed, to encourage usage and acceptability among patients and family members (56,57). Furthermore, accessibility and readability of information have increasingly become a focus for researchers and web designers, with the appreciation that even within a target population, there may be differing needs, abilities, and accessibility issues (58). Ascertaining the quality and authenticity of online information is also a significant issue for HNC patients (19). Within this current study, participants confirmed that knowing the online resource was being developed by HCPs with expert subject knowledge and researchers with previous patient-focused intervention development experience provided reassurance. The development of quality assessment tools such as DISCERN (59) (http://www.discern.org.uk) and HON (https://www.hon.ch/HONcode/Webmasters/Conduct.html) (60), and readability scales such as the Flesch Reading Ease Score (FRES) and the Flesch-Kinkaid Reading Grade Level (FKRGL) (61) should guide the development of web-based resources. Despite the availability of these tools, there is continuing evidence indicating a lack of well-designed web-based resources for HNC patients (62).

The current study also confirms the importance of qualitative data to inform intervention development, enhancing acceptability and usability in keeping with the recent MRC guidelines (20) and the person-based approach (21) to intervention development. Meaningful engagement with appropriate stakeholders at each phase of the intervention development is necessary to maximise the potential of developing or identifying an intervention that is likely to have positive impacts on behaviour change and health (21).

Limitations

The current study is limited to a specific patient cohort, as all patients had oral cancer with surgery as their primary treatment modality; therefore, the participants may not be representative of all HNC patients. Furthermore, the population of HNC survivors participating in this study also lacked cultural diversity, being a white British/Irish population, but were reflective of the current patient population within this particular service. Finally, due to the COVID-19 pandemic, all patient interviews took place remotely, mostly on the telephone at patients’ requests. As a result, this presented the researcher with the challenge of not being able to observe participant’s body language for discomfort or distress (63). This was addressed by regular member-checking throughout the interviews to ensure participants were content to continue. Of note, it appeared that patients preferred remote interviews as it minimised disruption to daily life. This is further supported by high research invitation uptake.


Conclusions

Consistent with current literature, there is an increasing recognition that high-quality, acceptable and engaging online resources for patients, necessitates a co-design process. Exploring and triangulating end-users (patients) and expert stakeholders (range of HCPs) perspectives in the planning phase (and indeed throughout all phases) of an online resource development for oral cancer patients can co-create more meaningful content. Content should be framed within a biopsychosocial lens, spanning three key landmarks (before, during and after treatments) of the cancer trajectory, encompassing patient experience narratives. To enhance acceptability, key design features such as layering and tailoring of information, plus easy navigational tools should be embedded. Using an intervention development framework such as the person-based approach or the new MRC guidelines can inform the development of online resources, which should enhance implementation and effectiveness in routine clinical care for patients.


Acknowledgments

The authors wish to thank all the staff and patients who participated in this phase of the study.

Funding: This study was supported by Macmillan Cancer Support, NI R&D Office, South Eastern Health & Social Care Trust.


Footnote

Provenance and Peer Review: This article was commissioned by Guest Editors (Simon N. Rogers, Anastasios Kanatas) for the series “Health-Related-Quality of Life in Head and Neck Oncology” published in Frontiers of Oral and Maxillofacial Medicine. The article has undergone external peer review.

Reporting Checklist: The authors have completed the COREQ reporting checklist. Available at https://fomm.amegroups.com/article/view/10.21037/fomm-22-1/rc

Data Sharing Statement: Available at https://fomm.amegroups.com/article/view/10.21037/fomm-22-1/dss

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://fomm.amegroups.com/article/view/10.21037/fomm-22-1/coif). The series “Health-Related-Quality of Life in Head and Neck Oncology” was commissioned by the editorial office without any funding or sponsorship. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). It was approved by the North West-Greater Manchester Central Research Ethics Committee (No. 19/NW/0720) and informed consent was taken from all individual participants.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Brady GC, Goodrich J, Roe JWG. Using experience-based co-design to improve the pre-treatment care pathway for people diagnosed with head and neck cancer. Support Care Cancer 2020;28:739-45. [Crossref] [PubMed]
  2. Fang CY, Galloway TJ, Egleston BL, et al. Development of a Web-Based Supportive Care Program for Patients With Head and Neck Cancer. Front Oncol 2020;10:602202. [Crossref] [PubMed]
  3. Kanatas A, Rogers SN. The role of the Head and Neck cancer-specific Patient Concerns Inventory (PCI-HN) in telephone consultations during the COVID-19 pandemic. Br J Oral Maxillofac Surg 2020;58:497-9. [Crossref] [PubMed]
  4. Chen ZJ, Liang WT, Liu Q, et al. Use of a Remote Oncology Pharmacy Service Platform for Patients With Cancer During the COVID-19 Pandemic: Implementation and User Acceptance Evaluation. J Med Internet Res 2021;23:e24619. [Crossref] [PubMed]
  5. Desai AV, Michael CL, Kuperman GJ, et al. A Novel Patient Values Tab for the Electronic Health Record: A User-Centered Design Approach. J Med Internet Res 2021;23:e21615. [Crossref] [PubMed]
  6. Warinner CB, Hayirli TC, Bergmark RW, et al. Patterns of Technology Use Among Patients With Head and Neck Cancer and Implications for Telehealth. OTO Open 2021;5:2473974X211018612.
  7. Musbahi A, Brown LR, Reddy A, et al. Systematic review of online patient resources to support shared decision making for bariatric surgery. Int J Surg 2020;74:34-8. [Crossref] [PubMed]
  8. Bond SM, Schumacher K, Dietrich MS, et al. Initial psychometric testing of the Head and Neck Cancer Patient Self-Management Inventory (HNC-PSMI). Eur J Oncol Nurs 2020;47:101751. [Crossref] [PubMed]
  9. McAlpine H, Joubert L, Martin-Sanchez F, et al. A systematic review of types and efficacy of online interventions for cancer patients. Patient Educ Couns 2015;98:283-95. [Crossref] [PubMed]
  10. Mihuta ME, Green HJ, Shum DHK. Efficacy of a web-based cognitive rehabilitation intervention for adult cancer survivors: A pilot study. Eur J Cancer Care (Engl) 2018;27:e12805. [Crossref] [PubMed]
  11. Nekhlyudov L, Lacchetti C, Davis NB, et al. Head and Neck Cancer Survivorship Care Guideline: American Society of Clinical Oncology Clinical Practice Guideline Endorsement of the American Cancer Society Guideline. J Clin Oncol 2017;35:1606-21. [Crossref] [PubMed]
  12. Smith AB, Bamgboje-Ayodele A, Butow P, et al. Development and usability evaluation of an online self-management intervention for fear of cancer recurrence (iConquerFear). Psychooncology 2020;29:98-106. [Crossref] [PubMed]
  13. Bruggeman-Everts FZ, Wolvers MDJ, van de Schoot R, et al. Effectiveness of Two Web-Based Interventions for Chronic Cancer-Related Fatigue Compared to an Active Control Condition: Results of the "Fitter na kanker" Randomized Controlled Trial. J Med Internet Res 2017;19:e336. [Crossref] [PubMed]
  14. Brebach R, Sharpe L, Costa DS, et al. Psychological intervention targeting distress for cancer patients: a meta-analytic study investigating uptake and adherence. Psychooncology 2016;25:882-90. [Crossref] [PubMed]
  15. Clover KA, Mitchell AJ, Britton B, et al. Why do oncology outpatients who report emotional distress decline help? Psychooncology 2015;24:812-8. [Crossref] [PubMed]
  16. Cnossen IC, van Uden-Kraan CF, Eerenstein SE, et al. A Participatory Design Approach to Develop a Web-Based Self-Care Program Supporting Early Rehabilitation among Patients after Total Laryngectomy. Folia Phoniatr Logop 2015;67:193-201. [Crossref] [PubMed]
  17. O'Connell Ferster AP, Hu A. Evaluating the quality and readability of Internet information sources regarding the treatment of swallowing disorders. Ear Nose Throat J 2017;96:128-38. [Crossref] [PubMed]
  18. Biggs TC, Jayakody N, Best K, et al. Quality of online otolaryngology health information. J Laryngol Otol 2018;132:560-3. [Crossref] [PubMed]
  19. Kelly R, Gordon P, Thompson R, et al. Availability and use of web-based interventions for patients with head and neck cancer: a scoping review. J Cancer Surviv 2022;28:1-18. [Crossref] [PubMed]
  20. Skivington K, Matthews L, Simpson SA, et al. Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update. Health Technol Assess 2021;25:1-132. [Crossref] [PubMed]
  21. Yardley L, Morrison L, Bradbury K, et al. The person-based approach to intervention development: application to digital health-related behavior change interventions. J Med Internet Res 2015;17:e30. [Crossref] [PubMed]
  22. INVOLVE. Pathways through participation: Research, engagement and impact. 2011. Available online: www.involve.org.uk
  23. Muller I, Rowsell A, Stuart B, et al. Effects on Engagement and Health Literacy Outcomes of Web-Based Materials Promoting Physical Activity in People With Diabetes: An International Randomized Trial. J Med Internet Res 2017;19:e21. [Crossref] [PubMed]
  24. Kitzinger J. The methodology of focus groups: the importance of interaction between research participants. Sociol Health Illn 1994;16:103-21. [Crossref]
  25. Cohen L, Manion L. Research Methods in Education. London: Croom Helm; 1986.
  26. Denzin N. The research act: A theoretical introduction to sociological methods. Chicago: Aldine Pub & Co.; 1970:368.
  27. Patton M.Q. Qualitative Research and Evaluation Methods. 3rd ed. Thousand Oaks: Sage Publications; 2002:598.
  28. Guba EG. Criteria for assessing the trustworthiness of naturalistic inquiries. Educ Technol Res Dev 1981;29:75-91.
  29. Mays N, Pope C. Quality in qualitative research. In: Pope C, Mays N. editors. Qualitative Research in Health Care. 2nd ed. London: BMJ Books; 2000:89-101.
  30. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007;19:349-57. [Crossref] [PubMed]
  31. Rabone C, Wallace V. A thematic analysis exploring the psychological well-being of adults born with esophageal atresia. J Psychosom Res 2021;145:110474. [Crossref] [PubMed]
  32. Braun V, Clarke V. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qual Res Psychol 2020;18:328-52. [Crossref]
  33. Ho KHM, Chiang VCL, Leung D. Hermeneutic phenomenological analysis: the ‘possibility’ beyond ‘actuality’ in thematic analysis. J Adv Nurs 2017;73:1757-66. [Crossref] [PubMed]
  34. World Medical Association [Internet]. Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects. 2013 (revised). Available from: www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/
  35. Brockbank S, Miller N, Owen S, et al. Pretreatment information on dysphagia: exploring the views of head and neck cancer patients. J Pain Symptom Manage 2015;49:89-97. [Crossref] [PubMed]
  36. Dornan M, Semple C, Moorhead A, et al. A qualitative systematic review of the social eating and drinking experiences of patients following treatment for head and neck cancer. Support Care Cancer 2021;29:4899-909. [Crossref] [PubMed]
  37. Semple CJ, Rutherford H, Killough S, et al. Long-term impact of living with an obturator following a maxillectomy: A qualitative study. J Dent 2019;90:103212. [Crossref] [PubMed]
  38. Crowder SL, Najam N, Sarma KP, et al. Quality of life, coping strategies, and supportive care needs in head and neck cancer survivors: a qualitative study. Support Care Cancer 2021;29:4349-56. [Crossref] [PubMed]
  39. Henry M, Alias A, Cherba M, et al. Immediate post-treatment supportive care needs of patients newly diagnosed with head and neck cancer. Support Care Cancer 2020;28:5557-67. [Crossref] [PubMed]
  40. Kim C, Dillon R, Nica L, et al. Information Needs of Patients Diagnosed with Head and Neck Cancer Undergoing Radiation Therapy: a Survey of Patient Satisfaction. J Cancer Educ 2021;36:941-9. [Crossref] [PubMed]
  41. Checklin M, Bain J, Bath L, et al. Patients' Perspectives on What Makes a Better Care Experience While Undergoing Treatment for Oropharyngeal Dysphagia Secondary to Head and Neck Cancer. Dysphagia 2020;35:702-16. [Crossref] [PubMed]
  42. Yang Y, Xu J, Hu Y, et al. The experience of patients with cancer on narrative practice: A systematic review and meta-synthesis. Health Expect 2020;23:274-83. [Crossref] [PubMed]
  43. Engler J, Adami S, Adam Y, et al. Using others' experiences. Cancer patients' expectations and navigation of a website providing narratives on prostate, breast and colorectal cancer. Patient Educ Couns 2016;99:1325-32. [Crossref] [PubMed]
  44. Ziebland S, Powell J, Briggs P, et al. Examining the role of patients’ experiences as a resource for choice and decision-making in health care: a creative, interdisciplinary mixed-method study in digital health. Programme Grants Appl Res 2016. doi: 10.3310/pgfar04170.10.3310/pgfar04170
  45. Lee CJ, Gray SW, Lewis N. Internet use leads cancer patients to be active health care consumers. Patient Educ Couns 2010;81:S63-9. [Crossref] [PubMed]
  46. Zhang Z, Yang H, He J, et al. The Impact of Treatment-Related Internet Health Information Seeking on Patient Compliance. Telemed J E Health 2021;27:513-24. [Crossref] [PubMed]
  47. DuBenske LL, Gustafson DH, Shaw BR, et al. Web-based cancer communication and decision making systems: connecting patients, caregivers, and clinicians for improved health outcomes. Med Decis Making 2010;30:732-44. [Crossref] [PubMed]
  48. Bailey SC, O'Conor R, Bojarski EA, et al. Literacy disparities in patient access and health-related use of Internet and mobile technologies. Health Expect 2015;18:3079-87. [Crossref] [PubMed]
  49. Zaharias G. What is narrative-based medicine? Narrative-based medicine 1. Can Fam Physician 2018;64:176-80. [PubMed]
  50. Pocock T, Smith M, Wiles J. Recommendations for Virtual Qualitative Health Research During a Pandemic. Qual Health Res 2021;31:2403-13. [Crossref] [PubMed]
  51. Semple CJ, Dunwoody L, George Kernohan W, et al. Changes and challenges to patients' lifestyle patterns following treatment for head and neck cancer. J Adv Nurs 2008;63:85-93. [Crossref] [PubMed]
  52. Hiatt JS, Young A, Brown TE, et al. Patient and carer experience of nutrition care throughout and beyond treatment for head and neck cancer: a qualitative longitudinal study. Support Care Cancer 2022;30:813-24. [Crossref] [PubMed]
  53. Schorn L, Lommen J, Sproll C, et al. Evaluation of patient specific care needs during treatment for head and neck cancer. Oral Oncol 2020;110:104898. [Crossref] [PubMed]
  54. Bingham SL, Semple CJ, Flannagan C, et al. Adapting and usability testing of an eLearning resource to enhance healthcare professional provision of sexual support across cancer care. Support Care Cancer 2022;30:3541-51. [Crossref] [PubMed]
  55. Rogers SN, Russell L, Lowe D. Patients' experience of temporary tracheostomy after microvascular reconstruction for cancer of the head and neck. Br J Oral Maxillofac Surg 2017;55:10-6. [Crossref] [PubMed]
  56. D'Souza V, Blouin E, Zeitouni A, et al. An investigation of the effect of tailored information on symptoms of anxiety and depression in Head and Neck cancer patients. Oral Oncol 2013;49:431-7. [Crossref] [PubMed]
  57. Lee K, Hoti K, Hughes JD, et al. Dr Google Is Here to Stay but Health Care Professionals Are Still Valued: An Analysis of Health Care Consumers' Internet Navigation Support Preferences. J Med Internet Res 2017;19:e210. [Crossref] [PubMed]
  58. Chee W, Lee Y, Chee E, et al. Practical guidelines for development of web-based interventions. Comput Inform Nurs 2014;32:504-11. [Crossref] [PubMed]
  59. Charnock D, Shepperd S, Needham G, et al. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53:105-11. [Crossref] [PubMed]
  60. Health on the Net (HON) HONcode Site Evaluation Form. 2009. Available online: https://www.hon.ch/HONcode/Webmasters/Conduct.html
  61. Kincaid JP, Fishburne RP Jr, Rogers RL, et al. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report. Millington, TN: Naval Technical Training Command, 1975:8-75.
  62. Chang KL, Grubbs EG, Ingledew PA AN. ANALYSIS OF THE QUALITY OF THYROID CANCER WEBSITES. Endocr Pract 2019;25:1003-11. [Crossref] [PubMed]
  63. Saarijärvi M, Bratt EL. When face-to-face interviews are not possible: tips and tricks for video, telephone, online chat, and email interviews in qualitative research. Eur J Cardiovasc Nurs 2021;20:392-6. [Crossref] [PubMed]
doi: 10.21037/fomm-22-1
Cite this article as: Semple CJ, Kelly R, Thompson R, Gordon P. Informing the development of an online resource for patients with oral cancer: triangulation of qualitative data from patients and healthcare professionals. Front Oral Maxillofac Med 2023;5:12.

Download Citation