Twitter Flu Trend: A Hybrid Deep Neural Network for Tweet Analysis

Abstract

Popular social networks such as Twitter have been proposed as a data source for public health monitoring because they have the potential to show infection disease surveillance like Influenza-Like Illnesses (ILI). However, shortness, data sparsity, informality, incorrect sentence structure, and the humorous are some challenges for tweet analysis and classification. In order to overcome these challenges and implement an accurate flu surveillance system, we propose a hybrid 1d-CNN-BiLSTM framework for semantic enrichment and tweet classification. Different embedding algorithms are compared for producing semantic representations of tweets to assist unrelated tweet filtering in the classification stage. We find that fine-tuning pre-trained Word2Vec enhances the model capability for representing the meaning of flu-related tweets than other embedding models. Our approach has been evaluated on a flu tweet dataset and compared with several baselines for tweet processing and classification. Experimental results show that: (1) the proposed hybrid deep neural networks can improve tweet classification due to considering their semantic information;(2) the proposed flu surveillance system achieves a state-of-the-art correlation coefficient with ILI rate published by CDC (https://www.cdc.gov/).