A decision support system for faculty performance management: A case report using statistical analysis, text mining, and artificial intelligence

Abstract

Purpose: This study presents a management methodology for comprehensively evaluating teaching performance by integrating statistical analysis of quantitative data, sentiment mining from text, and artificial intelligence tools. The objective is to provide academic managers with a robust and efficient diagnostic system that enables the continuous improvement of educational quality through the systematic identification of faculty strengths and areas for improvement, thereby facilitating the decision-making process in academic management.

Design/methodology/approach: The research adopts an Action Research approach, developing and implementing the EvalúaPro application using MATLAB® App Designer. Student evaluations from the 2425-1 (September-December 2024), 2425-2 (January-April 2025) and 2425-3 (April-July 2025)  academic periods were analyzed, which included quantitative (Likert scale questions) and qualitative (open-ended comments) components. For the 2425-1 period, 362 evaluations were analyzed, corresponding to 30 sections of 21 courses taught by 20 faculty members. For the 2425-2 period, 338 evaluations from 33 sections of 24 courses taught by 24 faculty members were processed, and for the 2425-3 period, 447 evaluations were analyzed, corresponding to 31 sections of 24 courses taught by 23 faculty members. All participants belonged to a department within the engineering faculty. Teaching competencies were strategically categorized into Soft Skills (Effective Communication, Interpersonal Skills, Time Management, and Organization) and Technical/Professional Skills (Content Mastery, Teaching Methodology). The qualitative analysis implemented the VADER algorithm for sentiment mining, while descriptive statistics were used for the quantitative analysis. Validation included tests with department heads to assess the application's effectiveness as a management tool.

Findings: The methodology proved highly effective for the managerial diagnosis of teaching performance, facilitating the identification of patterns at both individual and departmental levels. In the validation with department heads, 87.5% "agreed" or "strongly agreed" that the information presented by the prototype facilitates decision-making regarding faculty support, monitoring, and evaluation (37.5% "strongly agree," 50% "agree," 6.3% "neither agree nor disagree," 6.3% "strongly disagree"). Regarding the generated improvement plan, 93.8% of department heads "agreed" or "strongly agreed" that it accelerates feedback to faculty (43.8% "strongly agree," 50% "agree," 6.3% "strongly disagree"). Concerning its utility for diagnosis and decision-making for continuous improvement, 87.5% expressed they "agreed" or "strongly agreed" (62.5% "strongly agree," 25% "agree," 12.5% "neither agree nor disagree"). The system generated personalized improvement plans for faculty with scores below 3.0 and departmental strategies when more than 25% of professors showed similar areas for improvement. Furthermore, the system translates its integrated data analysis into a predictive tool, automatically alerting managers to signs of student dissatisfaction and thereby facilitating preemptive support measures.

Research limitations/implications: The main limitations include adapting the VADER algorithm for the specific academic context and requiring constant feedback to refine the artificial intelligence algorithms. Further research is required to validate the effectiveness of the automatically generated improvement plans in subsequent academic periods and their impact on improving teaching performance.

Practical implications: The methodology significantly reduces academic managers' time analyzing teaching evaluations, enabling faster and more specific feedback. The system facilitates identifying specific training needs that institutional resources, such as the Teaching Center, can address, thereby improving the efficiency of academic human resource management.

Social implications: Implementing this methodology enhances the analysis of educational evaluation, ensuring that student opinions are systematically considered for continuous institutional improvement, which can potentially reduce student attrition and enhance the overall educational experience.

Originality/value: This methodology represents an innovation that improves educational management by integrating advanced data analysis tools with structured managerial processes. The holistic approach, which combines statistical analysis, text mining, and artificial intelligence for faculty evaluation, offers significant value to educational institutions seeking to implement evidence-based continuous improvement systems. The strategic categorization of skills and the automatic generation of personalized improvement plans constitute an original contribution to educational management.