Improving stock price prediction using technical indicators and sentiment analysis

Document Type : Original Article

Authors

1 Ph.D. Student, Department of computer engineering, Yazd University, Yazd, Iran

2 Assistance Professor, Department of computer engineering, Yazd University, Yazd, Iran

3 Associated Professor, Department of computer engineering, Yazd University, Yazd, Iran

Abstract

With the increasing public use of social media platforms like Twitter, public opinion is influenced by posts shared by both influential individuals and the general public. These opinions and posts, particularly in the context of the stock market, guide traders in making buy or sell decisions. This study examines the impact of sentiment analysis on improving the accuracy of stock price prediction using six base estimators and five technical indicators. Experiments showed that selecting an appropriate time window (3, 7, 14, 30, 45, and 60 days) positively affects model performance. The prediction target was the stock price 7 days ahead. The results indicated that incorporating sentiment analysis improves prediction accuracy. The best-performing models were Linear Regression (LR), Multilayer Perceptron Regression (MLP), and Random Forest Regression (RF). Findings also revealed a correlation between tweet volume and trading volume, as well as a strong relationship between the average positive sentiment score in a day and the stock price. Specifically, based on the Mean Squared Error (MSE) evaluation metric, the best methods were LR, MLP, and RF. Stock price prediction improved by 17.37% using MLP and sentiment analysis, and by 34.78% using LR with sentiment analysis.

Keywords

References

[1] E. Ortiz-Ospina and M. Roser, "The rise of social media," Our World in Data, 2023. Available: https://ourworldindata.org/rise-of-social-media.
[2] M. M. P. Bharambe and S. C. Dharmadhikari, "Stock market analysis based on artificial neural network with big data," in Proc. 8th Post Graduate Conf. for Information Technology, 2017.
[3] J. J. Murphy, Technical analysis of the financial markets: A comprehensive guide to trading methods and applications, Penguin, 1999.
[4] B. M. Henrique, V. A. Sobreiro, and H. Kimura, "Literature review: Machine learning techniques applied to financial market prediction," Expert Syst. Appl., vol. 124, pp. 226–251, 2019.
[5] L. Z. Zhen, Y. H. Choo, A. K. Muda, and A. Abraham, "Forecasting FTSE Bursa Malaysia KLCI trend with hybrid particle swarm optimization and support vector machine technique," in Proc. 2013 World Congr. Nature Biol. Inspired Comput., IEEE, pp. 169–174, Aug. 2013
[6] O. Kraaijeveld and J. De Smedt, "The predictive power of public Twitter sentiment for forecasting cryptocurrency prices," J. Int. Financial Markets Inst. Money, vol. 65, 101188, 2020.
[7] A.Mohammadi, M. Yazdian Dehkordi, M. A. Nemat Bakhsh, (2019). Implicit feature detection using Persian language syntactic rules and feature clustering. Journal of Electrical Engineering, University of Tabriz. , Volume 50, Number 3, pp. 1395–1404, 2019.
[8] G. Ranco, D. Aleksovski, G. Caldarelli, M. Grčar, and I. Mozetič, "The effects of Twitter sentiment on stock price returns," PLoS One, vol. 10, no. 9, e0138441, 2015.
[9] M. Doğan, Ö. Metin, E. Tek, S. Yumuşak, and K. Öztoprak, "Speculator and influencer evaluation in stock market by using social media," in Proc. 2020 IEEE Int. Conf. Big Data, pp. 4559–4566, Dec. 2020.
[10] J. L. Bollen and H. Mao, U.S. Patent No. 8,380,607, Washington, DC: U.S. Patent and Trademark Office, 2013.
[11] X.-J. Zhang, "Twitter mood predicts the stock market," J. Comput. Sci., vol. 2, no. 1, pp. 1–8, 2011.
[12] P. C. Tetlock, M. Saar‐Tsechansky, and S. Macskassy, "More than words: Quantifying language to measure firms' fundamentals," J. Finance, vol. 63, no. 3, pp. 1437–1467, 2008.
[13] T. C. Lin, "The New Market Manipulation," Emory Law J., vol. 66, 2017.
[14] L. Phillips, C. Dowling, K. Shaffer, N. Hodas, and S. Volkova, "Using social media to predict the future: A systematic literature review," arXiv preprint arXiv:1706.06134, 2017.
[15] X. Li, H. Xie, L. Chen, J. Wang, and X. Deng, "News impact on stock price return via sentiment analysis," Knowl.-Based Syst., vol. 69, pp. 14–23, 2014.
[16] C. D. Kirkpatrick II and J. A. Dahlquist, Technical analysis: The complete resource for financial market technicians, FT Press, 2010.
[17] E. K. Sahin and S. Demir, "Greedy-AutoML: A novel greedy-based stacking ensemble learning framework for assessing soil liquefaction potential," Eng. Appl. Artif. Intell., vol. 119, 105732, 2023.
[18] Y. Xu, C. Yang, S. Peng, and Y. Nojima, "A hybrid two-stage financial stock forecasting algorithm based on clustering and ensemble learning," Appl. Intell., vol. 50, pp. 3852–3867, 2020.
[19] F. Moodi and H. Saadatfar, "An improved K-means algorithm for big data," IET Softw., vol. 16, pp. 48–59, 2021.
[20] M. M. Kumbure, C. Lohrmann, P. Luukka, and J. Porras, "Machine learning techniques and data for stock market forecasting: A literature review," Expert Syst. Appl., vol. 197, 116659, 2022.
[21] C. Xiao, W. Xia, and J. Jiang, "Stock price forecast based on combined model of ARIMA-LS-SVM," Neural Comput. Appl., vol. 32, pp. 5379–5388, 2020.
[22] N. Naik and B. R. Mohan, "Optimal feature selection of technical indicator and stock prediction using machine learning technique," in Proc. 2nd Int. Conf. Emerging Trends Comput. Electr. Eng. (ICETCE), Jaipur, India, pp. 261–268, Feb. 2019.
[23] G. Ji, J. Yu, K. Hu, J. Xie, and X. Ji, "An adaptive feature selection schema using improved technical indicators for predicting stock price movements," Expert Syst. Appl., vol. 200, 116941, 2022.
[24] A. U. Haq, A. Zeb, Z. Lei, and D. Zhang, "Forecasting daily stock trend using multi-filter feature selection and deep learning," Expert Syst. Appl., vol. 168, 114444, 2021.
[25] B. Weng, L. Lu, X. Wang, F. M. Megahed, and W. Martinez, "Predicting short-term stock prices using ensemble methods and online data sources," Expert Syst. Appl., vol. 112, pp. 258–273, 2018.
[26] U. Patel, "Twitter data predicting stock price using data mining techniques," M.S. thesis, Univ. of Bridgeport, 2016.
[27] P. Gao, R. Zhang, and X. Yang, "The application of stock index price prediction with neural network," Math. Comput. Appl., vol. 25, no. 3, 53, 2020. Available: http://dx.doi.org/10.3390/mca25030053
[28] D. V. Thanh, M. H. N. Minh, and D. D. Hieu, "Building unconditional forecast model of stock market indexes using combined leading indicators and principal components: Application to Vietnamese stock market," Indian J. Sci. Technol., vol. 11, no. 2, pp. 1–13, 2018. https://doi.org/10.17485/ijst/2018/v11i2/104908
[29] F. Moodi, A. Jahangard-Rafsanjani, & S. Zarifzadeh, "Feature selection and regression methods for stock price prediction using technical indicators," arXiv:2310.09903. https://doi.org/10.48550/arXiv.2310.09903.
[30] C. Hutto and E. Gilbert, "Vader: A parsimonious rule-based model for sentiment analysis of social media text," in Proc. Int. AAAI Conf. Web Social Media, vol. 8, no. 1, pp. 216–225, May 2014.
[31] Y. Sovbetov, "Factors influencing cryptocurrency prices: Evidence from bitcoin, ethereum, dash, litcoin, and monero," J. Econ. Financial Anal., vol. 2, no. 2, pp. 1–27, 2018.
[32] https://scikit-learn.org
[33] F. Li, W. Lu, J. W. Keung, X. Yu, L. Gong, and J. Li, "The impact of feature selection techniques on effort‐aware defect prediction: An empirical study," IET Softw., vol. 17, no. 2, pp. 168–193, 2023.
[34] A. M. Sefidian, N. Daneshpour, (2018). Applying Regression Models to High Correlation Subsets to Improve the Placement of Numerical Embedded Values. Journal of Electrical Engineering, University of Tabriz. Vol. 48, No. 3, pp. 1187–1200, 2018.
[35] R. K. Dash, T. N. Nguyen, K. Cengiz, and A. Sharma, "Fine-tuned support vector regression model for stock predictions," Neural Comput. Appl., pp. 1–15, 2021. https://doi.org/10.1007/s00521-021-05842-w.
Tabriz Journal of Electrical Engineering
Volume 55, Issue 2 - Serial Number 112
October 2025
Pages 357-370

Files

Share

How to cite

Statistics