The Integrated Management of Childhood Illness (IMCI) strategy, developed by the World Health Organization (WHO) and the United Nations Children’s Fund (UNICEF), was implemented in Brazil in 1996, proposing a comprehensive and systematic approach to child care, combining promotion, prevention, diagnosis, and treatment of prevalent childhood illnesses.⁽¹⁾In the Brazilian context, IMCI is linked to the National Policy for Comprehensive Child Health Care (PNAISC) as a guideline to improve care practices in the Health Care Network, starting from Primary Health Care (PHC).⁽²⁾

The systematic implementation of IMCI is associated with a reduction in infant mortality, greater adherence to clinical protocols, and rational use of medications.^(3,4) However, challenges persist for the strategy sustainability, including staff turnover, ongoing lack of supervision and support, and limited resources and reliance on printed materials, which make it difficult to standardize procedures and maintain quality of care.⁽⁵⁾

In this context, digital health technologies (mHealth) have emerged as tools capable of supporting clinical practice, strengthening continuing education processes, and promoting greater patient safety. Studies indicate that its adoption requires a prior assessment of usability, acceptability, and suitability to the healthcare context, especially when intended for large-scale use.^(6,7,8)

The digitization of clinical guidelines (including electronic versions of protocols such as IMCI) has been implemented in several countries, with evidence of improvements in assessment, classification, and clinical decision-making.^(9,10,11)

Usability, defined by the ISO 9241-11:2018 standard as the degree to which a system can be used by specific users to achieve objectives with effectiveness and satisfaction in a given context, is an essential element for the success of digital health technologies.⁽¹²⁾ Assessing the usability of a clinical app allows one to estimate adherence to real-world practice, acceptability by professionals, and potential impact on the safety and quality of care.^(13-15)

Recent studies on the use of clinical applications in child health (including tools such as the electronic Integrated Management of Childhood Illness (e-IMCI) and the Integrated eDiagnosis Approach, IeDA) have shown that digital support can increase diagnostic accuracy, reduce clinical errors, and improve the decision-making process in PHC.^{(10,16, 17)} However, empirical evidence regarding the performance and applicability of the IMCI application in Brazilian healthcare practice is still lacking, reinforcing the scientific and operational relevance of this investigation.

Therefore, the objective of this study was to describe the usability of the IMCI application among primary health care professionals, as measured by the System Usability Scale.

This descriptive study with a quantitative approach was conducted in the municipality of Arapiraca (Agreste region of Alagoas), Brazil. According to the Brazilian Institute of Geography and Statistics (IBGE; 2022 Census), this municipality has 234,696 inhabitants. The municipal primary health care network is composed of 38 Basic Health Units (BHUs), with 68 Family Health Teams (FHT), 7 Primary Care Teams (PCT), and 53 Oral Health Teams (OHT), ensuring 100% coverage. In 2021, a technical cooperation was established between the Pan American Health Organization (PAHO) and the Municipal Health Department in Arapiraca to qualify PHC professionals in the IMCI strategy; at that time, the principal researcher of this study acted as a certified multiplier and local coordinator of the implementation.

The study population comprised health professionals (nurses and physicians) certified in the operational course of the 2022 IMCI program and affiliated with the Municipal Health Department of Arapiraca. The sampling was intentional (non-probabilistic), considering the experience criteria in PHC and access to the application. The list of qualified professionals was obtained from the municipal coordination of the IMCI strategy and validated on the IMCI Digital Platform (Virtual Campus of Public Health, CVSP). Recruitment was carried out via electronic invitation letter and telephone contact, with authorization from the Directorate of Primary Care. Professionals with a degree in Nursing or Medicine, with ≥12 months of experience in PHC and access to a smartphone, were included. Professionals on leave, absence, reassignment, or who did not respond to the call were excluded. Of the 43 professionals qualified in the IMCI strategy, 28 were eligible, and 22 comprised the final sample.

The data was collected in person. The educational intervention was conducted in two classes (for logistical reasons), on June 20 and 21, 2023, during the working hours of the PHC units, with the consent of the respective directors. The activity included the following: dialogical theoretical presentation, demonstration of the functionalities of the IMCI application, download instructions, and simulation of two fictitious and representative clinical cases of prevalent childhood illnesses, developed according to the strategy’s protocol. During the simulations, participants performed the evaluation and classification steps, and defined clinical procedures with the help of the application, replicating the recommended decision-making flow. Usability was evaluated after the simulations were completed in order to capture the perception of recent use of the application.

The data collection instrument was composed of three sections. The first one covered the sociodemographic and professional characterizations, including education, length of time working in primary health care, prior use of the IMCI strategy in its printed version, and use of clinical applications geared towards professional practice. The second section assessed the application usability using the System Usability Scale (SUS), which was developed by Brooke (1996) and internationally validated as a fast, simple, and reliable method for measuring the usability of interactive systems. The cultural translation and adaptation of Tenório et al. (2011) was used, preserving semantic equivalence and psychometric validity. The scale consisted of 10 items arranged on a five-point Likert scale, ranging from “strongly disagree” to “strongly agree” (Chart 1).^(13,14,19)

Chart 1

Items	Original instrument in English	Consensual version in Portuguese
01	I think that I would like to use this system frequently	Acho que eu gostaria de usar esse sistema frequentemente
02	I found the system unnecessarily complex	Achei esse sistema desnecessariamente complexo
03	I thought the system was easy to use.	Pensei que esse sistema era fácil de usar.
04	I think that I would need the support of a technical person to be able to use this system	Acho que eu precisaria de ajuda técnica para poder usar este sistema.
05	I found the various functions in this system were well integrated.	Achei que as várias funções desse sistema estivessem bem integradas.
06	I thought there was too much inconsistency in this system	Achei que havia muita inconsistência neste sistema.
07	I would imagine that most people would learn to use this system very quickly.	Imagino que a maioria das pessoas pode aprender a usar esse sistema rapidamente.
08	I found the system very cumbersome to use.	Achei esse sistema muito pesado para usar.
09	I felt very confident using the system.	Eu me senti muito seguro usando o sistema.
10	I needed to learn a lot of things before I could get going with this system.	Preciso aprender muitas coisas antes de poder usar esse sistema.

The third section included an optional open-ended question to record comments, criticisms, and suggestions about the application. The inclusion of open-ended responses is recommended in the usability literature to support the interpretation of quantitative scores.⁽⁶⁾ The responses were anonymous and collected in an electronic form. The instrument was pre-tested by three professionals, who were not participants in the final sample, to verify the clarity and consistency of the items, and no adjustment was necessary.

The data were organized in an Excel spreadsheet and analyzed using R software (v. 4.0.2). Quantitative variables were described by means and standard deviations; categorical variables, by absolute and relative frequencies. The overall SUS score was calculated using the formula proposed by Brooke (1996) and converted to a 0-100 point scale. Interpretation followed the adjective classification of Bangor, Kortum, and Miller (2008), which categorizes scores as regular (<68.0), good (68.0-80.2), and excellent (≥80.3).⁽¹⁵⁾ Open-ended responses were analyzed descriptively to contextualize the quantitative results.

The study began after the signing of the Letter of Agreement from the Municipal Health Department and approval of the project by the Research Ethics Committee of the Federal Fluminense University (UFF) with the Certificate of Ethical Review Presentation (CAAE: 68316223.8.0000.5243) and Opinion # 6.110.881 (June 12, 2023), complying with the recommendations of the National Health Council Resolution (466/2012). All participants signed the Free and Informed Consent Form (FICF).

The sample consisted of 22 professionals, predominantly female (18; 81.8%) and with a Nursing degree (14; 63.6%). The majority were in the 25-39 age range (15; 68.2%). Regarding academic background, 17 (77.3%) participants had a specialization, and 11 (50%) professionals had less than 10 years of training. Regarding experience in PHC, 10 (45.5%) participants had ≥10 years of professional experience, and 17 (77.3%) had no other professional affiliation (Table 1).

Table 1

Variables	N=22 (%)
Age range (years)
25-39	15 (68.2)
≥40	7 (31.8)
Sex
Female	18 (81.8)
Male	4 (18.2)
Undergraduate course
Nursing	14 (63.6)
Medicine	8 (36.4)
Undergraduate institution
Public	10 (45.5)
Private	12 (54.5)
Graduation time (years)
<5	6 (27.3)
5-9	5 (22.7)
10-14	6 (27.3)
≥15	5 (22.7)
Academic qualification
Undergraduate degree	5 (22.7)
Specialization	17 (77.3)
Time working in PHC (years)
≥10	10 (45.5)
5-9	3 (13.6)
≤5	9 (40.9)
Do you have another professional affiliation?
Yes	5 (22.7)
No	17 (77.3)

PHC: Primary Health Care

Almost all participants (21; 95.5%) reported they needed a computer for work activities. Regarding applications geared towards clinical practice, users stated that they use them daily (15; 68.2%), occasionally (6; 27.3%), and do not use them (1; 4.55%) (Table 2).

Table 2

Variables	N=22 (%)
Computer use in the workplace
I partially need it	1 (4.5)
I need it	21 (95.5)
Experience with applications geared towards clinical practice
I do not use it	1 (4.5)
I use it, but not frequently	6 (27.3)
I use it daily	15 (68.2)

Regarding the use of the IMCI strategy (based on printed forms for assessing children under 5 years of age in PHC units), 1 participant reported not using them; most professionals (16; 72.7%) reported using them occasionally, whereas 5 (22.8%) reported doing so frequently.

The IMCI application obtained a mean score of 88.2±9.5 points on the SUS, corresponding to an excellent rating according to the adjective scale⁽¹⁷⁾. Scores ranged from 70.0-100 (all above the international cutoff point of 68.0), indicating a high level of usability and satisfaction among participating professionals.

Open comments indicated priorities for improvement: integration with the Electronic Citizen Record (ECR), speed in serious cases, desktop version, objectivity in nutrition and physical activity, storage of clinical records, and tutorial (Table 3); incompatibility in some Android models was the only inconsistency reported.

Table 3

Suggested Improvements	Participants	Total
Integration of the IMCI app with ECR	P3, P6, P8, P12, P18, P22	6
In urgent situations (classification: general danger and serious signs), expedite the assessment	P2, P7, P14, P16, P18, P20	6
Desktop version of the IMCI app	P2, P3	2
More objective assessment for physical activity	P16, P21	2
More objective assessment for nutrition	P16, P21	2
Store the clinical record in the IMCI app	P20	1
Tutorial on how to use the IMCI app	P5	1
Total suggested improvements: 7

The aspects related to practicality, ease of use, and objectivity of the consultations were highlighted among the compliments given by the participants. The professionals also emphasized the assertiveness of the clinical procedures, integration of functionalities, complete evaluation, and systematic organization of care provided by the application.

Other frequently mentioned points included technological excellence, lightweight application (low memory consumption on mobile devices), security in use, improved quality of care, and accessibility of the tool for daily use in PHC.

The findings indicated excellent usability of the IMCI application among the PHC professionals, with a rating above the international cutoff point of SUS, as proposed by Bangor, Kortum, and Miller.⁽¹⁵⁾This performance suggests high acceptability, ease of use, and functional consistency, reinforcing the application’s potential as a tool to support clinical decisions and standardize practices according to the IMCI strategy.

In light of the ISO 9241-11:2018 standard, usability could be translated as the system’s ability to enable users to achieve objectives with effectiveness, efficiency, and satisfaction.⁽¹²⁾ In this sense, the literature indicates that digital solutions applied to IMCI can optimize clinical reasoning, simplify workflows, and increase adherence to care guidelines.^(9-11,16,17)

International studies on the use of electronic algorithms in children’s healthcare (such as e-IMCI in Tanzania and IeDA in Burkina Faso) have shown greater adherence to clinical protocols and diagnostic accuracy, as well as a reduction in the unnecessary use of antibiotics.^(10,16,17)This evidence reinforces the potential of using clinical applications to qualify childcare, especially in low- and middle-income countries.

The qualitative compliments recorded in this study (such as practicality, ease of use, objectivity, integration of steps, and safety) reflect the classic usability dimensions described by Nielsen: ease of learning, efficiency, memorability, error reduction, and satisfaction.⁽²⁰⁾These findings are consistent with the application of SUS in the Brazilian context, which has already been validated and adapted to the local culture.^(19,14)

The suggestions for improvement pointed out by the participants (such as integration with ECR, availability of the desktop version, and inclusion of tutorials) reinforce the importance of approaches based on user-centered design and continuous usability evaluation; these principles are widely recommended in the development of digital health technologies.^(6,11)This approach contributes to sustainability, technological adherence, and continuous improvement of the user experience, which are essential for high-performance digital health systems.⁽⁸⁾

Incompatibility of the application with some Android smartphone models is the main inconsistency reported by participants. This finding agrees with the literature on mHealth: it has been pointed out that compatibility issues between devices, variations in performance, and interface barriers can reduce adoption and limit the impact of digital technologies in real-world care settings.^(6,7,9)

The use of the IMCI strategy in routine PHC showed to be limited among the professionals evaluated; a pattern was also described in different contexts. Studies have indicated that the complete implementation of the steps recommended by the WHO faces sustainability barriers even among trained professionals, compromising the standardization of management and the quality of care.^(4,5,17) Thus, ongoing education and systematic follow-up initiatives can strengthen the implementation of the strategy and increase the confidence to make clinical decisions when coupled with the use of clinical support applications (such as IMCI).

From a methodological point of view, the SUS scale is a valid, sensitive, and economical instrument for evaluating usability; however, it only offers global measurement, without detailing specific aspects of user-system interaction. The literature recommends combining different methods (such as structured observation, task testing, and qualitative interviews) to broaden the understanding of user experience.^(6,7,21)

Overall, the evaluation indicates high usability and good acceptability of the IMCI application among primary health care professionals, supporting its potential as a clinical and educational support tool. In addition to pointing out priorities for technical and organizational improvement, the body of evidence reinforces that well-structured digital solutions can contribute to improving child care and promoting greater adherence to clinical guidelines in PHC.^(8-10)

This study had limitations. The sample was intentional, restricted to a single municipality, and composed of a small number of participants, limiting generalization to other contexts. The evaluation was conducted after an educational intervention with clinical simulations (not considering the complexity of care in real-world conditions), which may influence the perception of usability. Furthermore, neither observational performance tests (e.g., task time and error rate) were applied nor were care outcomes evaluated, which prevents inferring clinical impact. Future multicenter studies (with observation of use in real-world scenarios) could deepen the understanding of the application’s effectiveness and incorporation in practice.

The Integrated Management of Childhood Illness (IMCI) application demonstrated excellent usability among Primary Health Care (PHC) professionals (average System Usability Scale score, SUS: 88.2), showing high satisfaction, ease of use, and functional consistency. The results point to its potential in supporting clinical and educational decisions to standardize practices and improve the quality of child care. There are now greater opportunities for improvement (integration with the Electronic Citizen Record, ECR, desktop version, compatibility between devices, and storage of forms and tutorials). This application is a viable and promising technology to strengthen children’s health care. In subsequent studies, we recommend evaluating its clinical effectiveness and incorporation into real-world practice.