Validity and reliability of the orthodontic experience questionnaire in Malayalam, an Indian language Jisha B, Sathyanadhan S, Sundareswaran S, Parayaruthottam P, Shibin M

ORIGINAL ARTICLE

Year : 2023 | Volume : 12 | Issue : 1 | Page : 48

Validity and reliability of the orthodontic experience questionnaire in Malayalam, an Indian language

Baby Jisha¹, Sreehari Sathyanadhan², Shobha Sundareswaran³, Prathapan Parayaruthottam¹, Muhammed Shibin¹
¹ Department of Orthodontics, Government Dental College, Kozhikode, Kerala, India
² Department of Orthodontics, Government Dental College, Thrissur, Kerala, India
³ Department of Orthodontics, KMCT Dental College, Mukkam, Kozhikode, Kerala, India

Date of Submission	08-Jan-2023
Date of Decision	16-Feb-2023
Date of Acceptance	10-Apr-2023
Date of Web Publication	04-Sep-2023

Correspondence Address:
Baby Jisha
Department of Orthodontics, Government Dental College, Kozhikode, Kerala - 673 008
India

Source of Support: None, Conflict of Interest: None

Check

DOI: 10.4103/jos.jos_5_23

Abstract

AIM: To translate the orthodontic experience questionnaire into Malayalam, an Indian language, and assess its validity and reliability in the Kerala population.
MATERIALS AND METHODS: The original questionnaire, which was developed in English, was translated into Malayalam. Face validity was assessed, and minor changes were made, followed by a content validity assessment by an expert panel. The final version was given to 324 patients who marked their responses on a Likert scale. Internal consistency reliability was tested using Cronbach's alpha, and test-retest reliability was tested using the intraclass correlation coefficient. Principal component factor analysis was performed to regroup the valid items.
RESULTS: The principal component factor analysis revealed three domains. Reliability assessed by Cronbach's alpha of the whole questionnaire was 0.799, and that of the three domains was 0.73, 0.71, and 0.71, respectively. The test-retest reliability was found to be good (0.798).
CONCLUSION: The Malayalam version of the orthodontic experience questionnaire was found to be content valid with good reliability. To obtain a successful treatment result, an orthodontist must be aware of these and this questionnaire, with adequate validity and reliability, has been designed to achieve this in the Malayalam-speaking population.

Keywords: Fixed appliance, content validity, orthodontics, questionnaire validation, reliability, treatment experience

How to cite this article:
Jisha B, Sathyanadhan S, Sundareswaran S, Parayaruthottam P, Shibin M. Validity and reliability of the orthodontic experience questionnaire in Malayalam, an Indian language. J Orthodont Sci 2023;12:48

How to cite this URL:
Jisha B, Sathyanadhan S, Sundareswaran S, Parayaruthottam P, Shibin M. Validity and reliability of the orthodontic experience questionnaire in Malayalam, an Indian language. J Orthodont Sci [serial online] 2023 [cited 2023 Sep 21];12:48. Available from: https://www.jorthodsci.org/text.asp?2023/12/1/48/385072

Introduction

Orthodontic treatment primarily aims at improving facial esthetics by addressing skeletal, dental, and functional derangements. Patients' psychosocial well-being is of paramount importance when considering the goals of orthodontic treatment.^[1],[2] Fixed orthodontic therapy has been the mainstay treatment for malocclusion for decades. A wide range of malocclusions can be treated using these appliances, particularly those that cannot be treated with removable appliances. Physical and mental discomfort during fixed orthodontic treatment, such as pain, soreness of teeth, and general embarrassment, can negatively impact patients' self-esteem.^[3]

Most published studies in the literature have used questionnaires designed for children using oral health-related quality of life (OHRQoL) questionnaires or their modifications.^[4] These questionnaires were aimed at understanding the impact of malocclusion on patients' quality of life without focusing on the treatment of malocclusions and associated problems. However, little attention has been paid to studying the impact of and experiences with fixed appliance treatment.

O'Brien et al.^[5] developed a set of questionnaires to evaluate the treatment experiences of patients wearing functional appliances. Yassir et al.^[6] (2016) adopted and modified these questionnaires to assess fixed appliances. They developed a series of three questionnaires for measuring patient expectations (pre-treatment questionnaire), experience (orthodontic experience questionnaire), and contentment (post-treatment questionnaire) with fixed appliance orthodontic treatment. In a developing country like India, as people embrace newer treatment options, it becomes mandatory to know their perceptions regarding these options.

Because most of the questionnaires relevant to this context originated in English-speaking countries and are written in English, they need translation into local languages and validation by the local population.^[7] Kerala, a southern state in India, has a higher literacy rate than other states.^[8],[9] Due to this factor, the effect of health and psychological factors attains a dominant position in this population.^[10],[11] Under these circumstances, data on how individuals perceive fixed orthodontic appliances as a means for correcting their malocclusion and their experiences after a period of appliance wear are important. Hence, this study aimed to translate the orthodontic experience questionnaire into Malayalam, the official language of Kerala, and validate it in the North Kerala population.

Materials and Methods

The English version of the orthodontic experience questionnaire was developed in the United Kingdom by Yassir et al.^[6] and validated in the local population who were undergoing fixed orthodontic therapy. Hence, the questionnaire requires translation into the local language to be applicable to the concerned population.^[12] Approval for the study was obtained from the institutional research committee (protocol reference no: 143/2018/DCC). The original instrument consisted of two domains and a total of 18 items. The researchers of the original instrument were informed prior to the initiation of the study.

Sample

A convenience sample was selected from young adults undergoing regular fixed orthodontic therapy at the Department of Orthodontics. We decided to include willing orthodontic patients aged 18–30 years who were undergoing treatment for 3–6 months. The exclusion criteria were a) any mental or behavioral disorder that reduced the ability for self-determination and b) craniofacial anomalies. The sample size was determined to provide an 80% statistical power with an alpha level of 0.05 and was found to be 320.

Translation and back translation

The original instrument was translated into Malayalam by two translators who were bilingual, i.e., proficient in both the source and target languages; one was an orthodontist and the other was a clerk. The translation was performed while keeping in mind the local vocabulary and jargon. This requires that the translators have an in-depth understanding of both cultures. The Malayalam version developed was carefully back-translated to English by an orthodontist who was bilingual and an English teacher who did not have in-depth knowledge of the meaning of the items. The resulting instrument and the original were then compared.^[13],[14] Certain items needed revision; hence, all the above steps were repeated until conceptual and semantic equivalence was achieved.

Content validity

Face validity was assessed by distributing the questionnaire to determine whether the items were clear and comprehensible to the target population. Minor changes in wording were made, which made the items compatible with the local vocabulary. The final version was content-validated by a panel comprising 10 members: five orthodontists with more than 5 years of clinical experience and five orthodontic post-graduate students.^[15] The orthodontists in the panel had wide experience as they were practicing in different parts of the state. Copies of the questionnaire were distributed to each panel member, and each item was assessed carefully.

Cultural adaptation

Cultural adaptation was achieved with the help of another committee comprising three orthodontists who were working in nearby clinics and a community group in the district. They evaluated the questionnaire to determine whether the various concepts were relevant to the cultural context of this society. Conceptual and semantic equivalence was assessed, and proposed modifications were made to improve the overall appearance of the questionnaire.

Pilot study

This was followed by pilot testing conducted on a sample of 30 patients aged 18–30 years comprising 15 females and 15 males whose mother tongue was Malayalam, the target language. The pilot study was conducted by a single investigator by means of direct interviews. The goal was to assess any difficulties in comprehending the questionnaire. At the end of this phase, appropriate alterations were made, and the final version of the Malayalam questionnaire was obtained.

The final version of the questionnaire was distributed to 324 participants who provided informed consent. The responses were recorded by the first author on a 5-point Likert scale: 1: strongly disagree, 2: disagree, 3: uncertain, 4: agree, and 5: strongly agree. The time taken to complete the questionnaire was approximately 10–12 minutes.

Statistical analysis

Data obtained from the respondents were evaluated using Statistical Package for Social Sciences software (SPSS Version 25.0, IBM Corp., Armonk, NY) considering alpha = 0.05. The scores were summed for each domain by considering the response options. Missing data were excluded because the responses were marked by the first author.

Psychometric evaluations included internal consistency/reliability measures using Cronbach's alpha and test-retest reliability using the intraclass correlation coefficient. The Kaiser-Meyer-Olkin (KMO) test of sampling adequacy and Bartlett's test were performed to check whether the data were suitable for principal component analysis (PCA), which was done for item reduction to obtain a questionnaire with increased sensitivity.

Results

There were a total of 324 participants, of which 63.6% were females. Within each domain, there was no statistically significant difference [Table 1]. Normality was assessed using Shapiro-Wilk test and the data was found to have a normal distribution, P > 0.05.

Table 1: Demographic characteristics of the participants

Click here to view

The internal consistency reliability of the overall questionnaire was 0.799, which is considered good. The reproducibility, assessed as test-retest reliability using the intraclass correlation coefficient, was 0.798.

The final instrument consisted of 13 items, and the result were acceptable; an item related to changes in eating habits, with a factor loading of 0.814, was not correlated with other groups. The responses to this item were reversed; thus, it could be added to the first group. Hence, the first group included eight items measuring psychological impact and function. This involved 26.23% of variance, with an eigenvalue of 3.9. The internal consistency of this group was 0.729, which was appropriate. The second group consisted of three items that measured the impact of braces on school/work and interpersonal relationships. This group had 13.73% of variance and an eigenvalue of 2.06 and Cronbach's alpha of 0.712. The third group included two items measuring self-confidence, which showed a variance of 8.0% and an eigenvalue of 1.2. The internal consistency for this group was 0.706.

A KMO test score of 0.776 and Bartlett's test with P value < 0.05 (0.001) indicates good correlation in the data. Hence, PCA could be conducted. Three groups emerged in the PCA. The factor loadings are depicted in [Table 2]. The final questionnaire is shown in [Figure 1].

Figure 1: Validated Malayalam version of the Orthodontic experience questionnaire consisting of three domains and thirteen items

Click here to view

Table 2: Reliability and factor loading of the three domains emerged from principal component factor analysis

Click here to view

Discussion

The orthodontic experience questionnaire, translated and validated in this study, is one of its kind as it provides insight into the psychological aspect of a patient's experience while on fixed appliance treatment. This is the key factor that motivates patients to maintain their braces and continue treatment. Although this tool was developed from a questionnaire designed to evaluate the patient perception of fixed orthodontic treatment in the United Kingdom, most of the items from the original questionnaire were found to be relevant and, hence, retained. Unlike indices developed from a clinician's viewpoint, this tool focuses on problems faced by patients and their physical and mental discomfort during treatment, such as pain, eating habits, social relationships, and issues related to school/workplace.^[16],[17]

The tool's face validity was found to be adequate. However, because it is a weak measure of validation, content validation was performed. The content validity of the questionnaire was determined based on the combined opinions and suggestions of experts in the field. The level of experience and expertise in the specific content area ensures that the items being studied are relevant to the specific population. Hence, the panel consisted of five members with more than 5 years of experience in the field of orthodontics and five post-graduate students who were natives and well-acquainted with patients from the local population through daily interactions.^[15] After the first round of validation, only minor changes were required. Therefore, a second round of validation was avoided.^[15],[18] Two items, i.e., getting bullied/teased by friends and a similar item related to family owing to the presence of braces in the mouth, were clubbed into one as recommended by experts. Similarly, two items pertaining to changes in appearance affecting a person's interactions with friends and that with family were clubbed into a single item: changes in appearance affecting interpersonal relationships. The item “overall experience,” which was present in the original instrument, was removed because of low validity, as agreed upon by the content reviewers. Since most of the individual items were related to various aspects of the patients undergoing fixed appliance treatment, the question of overall experience seems to make it difficult for the patient to mark the response, which may in turn prove to be spontaneous or impulsive. An item related to cleaning of braces, which was eliminated in the original instrument, was added to this version. This was based on the fact that there is a high incidence of dental caries among individuals of the given population and the incidence of caries during the course of fixed orthodontic treatment is also high.^[19],[20]

The original instrument included 16 items divided into two domains: i) function, self-concept, and interpersonal relations, and ii) pain and experience of wearing braces. Based on PCA, three domains were obtained: i) psychological effects and function, ii) work and interpersonal relationships, and iii) self-confidence. “Changes in eating habits” was initially a separate item, which could not be placed in the first three domains as it was not correlated to those constructs. However, upon item reversal, it could be grouped under Domain 1; hence, this domain was named “psychological effects and function.”^[21] Internal consistency reliability was assessed using Cronbach's alpha. The whole questionnaire had an internal consistency score, which was adequate (0.79). Domains 1, 2, and 3 also showed good internal consistency. Hence, it can be deduced that the questionnaire was an adequate measure of its construct. Domain 3 included two questions: changes in bullying/teasing after wearing braces and whether the patient is bothered about extra appointments due to bracket breakage. These questions were combined under the domain name “self-confidence.” In most cases, during the first 3 months of treatment, anterior tooth alignment would have improved compared with the pre-treatment condition, which greatly enhances the patient's confidence.^[3] Hence, bullying due to unesthetic faces and teeth would have been reduced. It is also possible that the impact of bullying on patients may have been curtailed owing to improved self-esteem. Since the patients from whom responses were collected were those who were undergoing treatment for 3–6 months, the initial difficulties after placement of brackets would have reduced, and they would have adapted well to the new oral environment.^[22],[23] Consequently, bracket breakage also seems to decrease over time.

Each domain of this questionnaire exhibited good reliability; hence, it can be used separately. Questions 1–8 (Domain 1) can be used to assess the psychological impact of braces, which provides insight into how the treatment affects a patient from their own perspective.^[24] This is important because the manner in which patients adapted to the appliance and how they tackled problems might have more to it than what is actually observed in the clinic. Hence, this tool can be a valuable addition for orthodontists in understanding the psychological aspects of patients undergoing treatment and providing guidance if needed.

Comparison with previous questionnaires

Similar questionnaires by Mandall et al. and Yassir et al., the latter being the original instrument (orthodontic experience questionnaire) from which the present tool was created, have been developed in the United Kingdom.^[6],[25] Although great cultural differences exist, most of the items in the original questionnaire were retained. The items related to cleaning the brace and eating habits, which were omitted from the orthodontic experience questionnaire owing to low validity, were added back owing to their relevance in the study population. The tool developed by Mandall et al. was aimed at 10–18-year-old orthodontic patients, whereas our questionnaire was developed for young adults aged 18–30 years. Another apparent difference was the timing of administration of the questionnaires. Previous questionnaires were administered to patients who were at various stages of orthodontic therapy. However, our questionnaire was distributed to patients who had undergone 3–6 months of orthodontic treatment, considering that they would have overcome the initial difficulties and adjusted to the appliance.^[22],[23]

The limitation of the study is that it was conducted in an institution in the public health sector, where a major chunk of the patients who seek treatment are from weak socioeconomic backgrounds. Hence, the treatment priorities and expectations may not match those of the general population.

Clinical significance

Patients undergoing fixed orthodontic treatment experience physical, psychological, and social problems. To obtain a successful treatment result, an orthodontist must be aware of these problems, and this tool, with adequate validity and reliability, has been designed to achieve this in the Malayalam-speaking population.

Conclusion

The translated version of the orthodontic experience questionnaire was found to be psychometrically sound and could be used in the Malayalam-speaking population.
This tool has good validity and internal consistency reliability, making it useful for evaluating the treatment experiences of patients undergoing fixed orthodontic therapy.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

1.	Cunningham SJ, Hunt NP. Quality of life and its importance in orthodontics. J Orthod 2001;28:152-8.
2.	Deng X, Wang YJ, Deng F, Liu PL, Wu Y. Psychological well-being, dental esthetics, and psychosocial impacts in adolescent orthodontic patients: A prospective longitudinal study. Am J Orthod Dentofac Orthop 2018;153:87-96.
3.	Johal A, Alyaqoobi I, Patel R, Cox S. The impact of orthodontic treatment on quality of life and self-esteem in adult patients. Eur J Orthod 2015;37:233-7.
4.	Salim Z, Majid A, Abidia Bds RF, Rcs D. Effects of malocclusion on oral health related quality of life: A critical review. Eur Sci J 2015;11:386-400.
5.	O'Brien, K, Kay, L, Fox D MN. Assessing oral health outcomes for orthodontics–measuring health status and quality of life. Community Dent Health1998;15:22-6.
6.	Yassir YA, McIntyre GT, Bearn DR. Three questionnaires to assess the perception of fixed orthodontic therapy before, during and after treatment: Validity and reliability. Eur J Orthod Aug 2017;39:402-10.
7.	Montiel-Company JM, Bellot-Arcís C, Almerich-Silla JM. Validation of the psychosocial impact of dental aesthetics questionnaire (Pidaq) in Spanish adolescents. Med Oral Patol Oral Cir Bucal 2013;18:e168.
8.	Ranjan M, Dwivedi LK. Infant mortality and death clustering at the district level in India: A Bayesian approach. Spat Spatiotemporal Epidemiol 2022;41:100481.
9.	Swargiary K Roy. Literacy rate in india in 2022. Acad An Int Multidiscip Res J 2022;12:87-93.
10.	Hone P, Black J, Sathish T, Kapoor N, Cao Y, Haregu T, et al. Determinants of health service utilization among adults at high risk of developing type 2 diabetes in Kerala, India. Asia Pacific J Public Heal 2022;34:377-83.
11.	Muttappallymyalil J, Divakaran B, Sreedharan J, Salini K SS. Oral health behaviour among adolescents in Kerala, India. Ital J Public Health 2012;6:218-24.
12.	Francis Guillemin, Claire Bombardier DB. Cross-cultural adaptation of health-related quality of life measures : Literature review and proposed guidelines. J Clin Epidemiol1993;46:1417-32.
13.	Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: A clear and user-friendly guideline. J Eval Clin Pract 2011;17:268-74.
14.	Gjersing L, Caplehorn JR, Clausen T. Cross-cultural adaptation of research instruments: Language, setting, time and statistical considerations. BMC Med Res Methodol 2010;10:13.
15.	Lynn M. Determination and quantification of content validity. Nurs Res 1986;35:382-85.
16.	Richmond S, Shaw WC, O'brien KD, Buchanan IB, Jones R, Stephens CD, et al. The development of the PAR Index (Peer Assessment Rating): Reliability and validity. Eur J Orthod 1992;14:125-39.
17.	Richmond S, Shaw WC, Roberts CT AM. The PAR Index (Peer Assessment Rating): Methods to determine outcome of orthodontic treatment in terms of improvement and standards. Eur J Orthod 1992;14:180-7.
18.	Polit DF, Beck CT, Owen S V. Focus on research methods: Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Heal 2007;30:459-67.
19.	Sreela LS, Balan A, Peter E, M BR, Narayan V, Nair AK. Association between Oral Health Status and Quality Of Life in Kerala: A population based cross-sectional study. J Adv Med Dent Sci Res 2020;8:49-55.
20.	Haridas H, KS Premlal MS. Study on dental caries and risk factors for non communicable diseases among undergraduate medical students of Palakkad district, Kerala. Indian J Public Heal Res Dev 2020;11:44-9.
21.	Elbe Peter, R. M. Baiju, N. O. Varghese, Remadevi Sivaraman DLS. How to develop and validate a questionnaire for orthodontic research. Eur J Dent 2017;11:411-6.
22.	Hamid Rakhshan VR. Pain and discomfort perceived during the initial stage of active fixed orthodontic treatment. Saudi Dent J 2015;27:81-7.
23.	Noorhanizar Mansor, Roslan Saub SAO. Changes in the oral health-related quality of life 24 h following insertion of fixed orthodontic appliances. J Orthod Sci 2012;1:98-102.
24.	Feldmann I, List T, John MT, Bondemark L. Reliability of a questionnaire assessing experiences of adolescents in orthodontic treatment. Angle Orthod 2007;77:311-7.
25.	Mandall NA, Vine S, Hulland R, Worthington H V. The impact of fixed orthodontic appliances on daily life. Community Dent Health 2006;23:69-74.