Automated ticket classification for training ISTAT's PUC chatbot

Abstract

The increasing volume of user requests handled by ISTAT’s contact center for supporting participants in official statistical survey has underscored the need for automated solutions to optimise ticket classification and reduce reliance on manual processing.

This study presents the development and evaluation of a supervised classification system that leverages Natural Language Processing (NLP) techniques to enhance the accuracy, efficiency and scalability of request management within a public administration context.

The proposed framework integrates a TF-IDF-based text representation with synthetic oversampling (SMOTE) and three supervised learning algorithms: Random Forest, LightGBM, and Multilayer Perceptron. The methodology also incorporates a tailored preprocessing pipeline—covering tokenisation, lemmatisation, stopword removal, and anonymisation of personal information—to ensure data quality and privacy compliance.

The classification system was designed to support the training phase of ISTAT’s PUC chatbot, which will provide first-level assistance to citizens and establishments involved in statistical surveys. By generating high-quality labelled data, this approach aims to improve chatbot intent recognition and facilitate self-service interactions for survey respondents.

Model performance was evaluated using standard classification metrics, including accuracy and both weighted and macro-averaged F1 scores.

Among the tested configurations, LightGBM demonstrated the most balanced and robust performance. The results confirm the effectiveness of integrating machine learning and NLP into institutional workflows. Future work will explore the integration of the classifier into the generative architecture of Salesforce Agentforce, contributing to the evolution of intelligent support systems in citizen-facing public services.