نوع مقاله : علمی-پژوهشی
نویسندگان
1 دانشگاه تربیت مدرس تهران
2 دانشکده مهندسی برق و کامپیوتر، دانشگاه تربیت مدرس، تهران، ایران
چکیده
کلیدواژهها
عنوان مقاله [English]
نویسندگان [English]
User stance detection means determining the user's attitude (agree, disagree, or neutral) towards a particular issue. Academic and industry research is highly interested in the automatic analysis of opinions in social networks. One common method for training efficient models is knowledge distillation, in which knowledge is transferred from a large, complex model (the teacher) to a smaller, lighter model (the student).
Calibration refers to the degree to which the model’s confidence aligns with its actual accuracy. In previous methods, using a single calibrated teacher in a multi-generational framework causes error propagation due to its dependence on the previous generation, resulting in unstable training. To address this challenge, ensemble calibrated knowledge distillation is proposed. In the proposed framework, a dynamic ensemble of the best calibrated models from all previous generations is used, which makes the training process more robust and diverse. In addition, a self-paced calibration annealing strategy is introduced, which, by applying a simple calibration objective in the early stages, helps in stable feature learning and focuses on more precise error optimization in the later stages. The results evaluated based on F-micro and F-macro metrics show an improvement of at least 3% compared to the baseline calibrated model on different real-world datasets (COVID-19, P-Stance, and AM).
کلیدواژهها [English]
)