A HYBRID NLP AND CLUSTERING-BASED FRAMEWORK FOR INDUSTRY-ALIGNED ACADEMIC COURSE RECOMMENDATIONS

Abstract

The increasing mismatch between academic programs and the fast changing industry needs pose a major predicament to institutions of higher learning that want to churn out employment-worthy graduates. In this paper, we suggest a hybrid recommendation system composed of Natural Language Processing (NLP), unsupervised clustering, topic modeling and similarity analysis to support learning by matching course content in the university with up-to-date trends in the job market. TF-IDF vectorization was applied and Dice.com data of more than 25,000 job descriptions was used to perform K-Means clustering for grouping job roles in thematic clusters. Dimensionality reduction and visualization of clusters was carried out using Principal Component Analysis (PCA) and dominant skill-based topics within groups were found using the Latent Dirichlet Allocation (LDA). By cosine similarity, these topics were aligned with the academic course outlines to determine similarity. It was found in the experiments that topics were highly semantically coherent (all with a score of more than 0.5) and the cosine similarity between courses and topics showed extreme scores (more than 0.6). The highest similarity (0.742) occurred between Data Analysis and Data Science. There is a great potential in applying the proposed system to cover the academic and enterprise divide through the implementation of data-driven dynamic course creation to meet the current workforce needs.