An Adaptive Computational Model for Detecting Concept Drift in Long Term Data Streams Using Incremental Learning Approaches
Keywords:
Concept Drift, Incremental Learning, Drift Detection, Model Efficiency, Real Time AdaptabilityAbstract
Concept drift, the phenomenon where the statistical properties of data streams change over time, poses a significant challenge in machine learning, particularly for long term data streams. Traditional machine learning models, including batch learning and non-adaptive approaches, struggle to detect and adapt to these changes, leading to degraded performance and inaccurate predictions. This study proposes an adaptive computational model designed to detect and respond to concept drift using incremental learning techniques and statistical drift detection mechanisms. The model integrates an Adaptive Drift Detector (ADD) and Incremental Learning System, enabling real-time adjustments to data distribution changes. The model is evaluated across synthetic and real-world datasets, demonstrating its superior ability to detect abrupt, gradual, and recurring drifts compared to traditional models. Experimental results indicate that the adaptive model maintains high prediction accuracy, minimizes false positive rates, and reduces detection delays. Furthermore, the model performs well in resource-constrained environments, making it suitable for real-time applications such as healthcare prediction, fault detection, and IoT systems. Despite its promising performance, the study identifies challenges related to computational complexity and the model’s performance with imbalanced datasets and noisy data. Future research should focus on optimizing the model’s scalability, computational efficiency, and adaptability to more complex data types to ensure broader applicability in dynamic environments. This work contributes to advancing the detection and adaptation of concept drift, offering a robust solution for dynamic and evolving data streams.
References
[1] G. J. Aguiar and A. Cano, “A comprehensive analysis of concept drift locality in data streams,” Knowledge-Based Syst., vol. 289, 2024, doi: 10.1016/j.knosys.2024.111535.
[2] M. S. Abdul Razak, C. R. Nirmala, B. R. Sreenivasa, L. Husam, and L. Hassan Fareed M., “A survey on detecting healthcare concept drift in AI/ML models from a finance perspective,” Front. Artif. Intell., vol. 5, 2022, doi: 10.3389/frai.2022.955314.
[3] S. Suryawanshi, A. Goswami, and P. Patil, “IRBM: Incremental Restricted Boltzmann Machines for Concept Drift Detection and Adaption in Evolving Data Streams,” Commun. Comput. Inf. Sci., vol. 2053 CCIS, pp. 466 – 475, 2024, doi: 10.1007/978-3-031-56700-1_37.
[4] L. Korycki and B. Krawczyk, “Concept drift detection from multi-class imbalanced data streams,” in Proceedings - International Conference on Data Engineering, 2021, pp. 1068 – 1079. doi: 10.1109/ICDE51399.2021.00097.
[5] M. Klaiber, M. Rössle, and A. Theissler, “The 10 most popular Concept Drift Algorithms: An overview and optimization potentials,” in Procedia Computer Science, 2023, pp. 1261 – 1271. doi: 10.1016/j.procs.2023.10.114.
[6] M. A. Hashmani, S. M. Jameel, M. Rehman, and A. Inoue, “Concept Drift Evolution In Machine Learning Approaches: A Systematic Literature Review,” Int. J. Smart Sens. Intell. Syst., vol. 13, no. 1, pp. 1 – 16, 2020, doi: 10.21307/ijssis-2020-029.
[7] L. Baier, V. Kellner, N. Kühl, and G. Satzger, “Switching scheme: A novel approach for handling incremental concept drift in real-world data sets,” in Proceedings of the Annual Hawaii International Conference on System Sciences, 2021, pp. 990 – 999. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85108373607&partnerID=40&md5=b0e106887ec3fdb2d05d6d9326f161f4
[8] P. Zhou and Y. Zhang, “A review of concept drift detection and adaptation methods from an industrial perspective; [基于工业视角的概念漂移检测与适应方法综述],” Kongzhi yu Juece/Control Decis., vol. 40, no. 6, pp. 1774 – 1792, 2025, doi: 10.13195/j.kzyjc.2024.1431.
[9] F. Giannini and E. Della Valle, “Addressing Temporal Dependence, Concept Drifts, and Forgetting in Data Streams,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 15013 LNCS, pp. 146 – 164, 2025, doi: 10.1007/978-3-031-82346-6_10.
[10] G. V Prasad and K. Sharma, “A Unified Framework for Detecting Gradual and Abrupt Concept Drifts in Data Stream Mining: The Concept Drift Detection Framework with Hybrid Meta-Learning (CDDF-HML),” SSRG Int. J. Electr. Electron. Eng., vol. 11, no. 7, pp. 39 – 50, 2024, doi: 10.14445/23488379/IJEEE-V11I7P103.
[11] N. Nagendran, H. P. Sultana, and A. Sarkar, “A comparative analysis on ensemble classifiers for concept drifting data streams,” SpringerBriefs Appl. Sci. Technol., pp. 55 – 62, 2019, doi: 10.1007/978-981-13-0059-2_7.
[12] Y. Chen and W. Guo, “Concept drift data stream regression model based on adaptive drift detection and incremental broad learning,” in Proceedings of SPIE - The International Society for Optical Engineering, 2025. doi: 10.1117/12.3061638.
[13] H. Gâlmeanu and R. Andonie, “Concept drift adaptation with incremental–decremental svm,” Appl. Sci., vol. 11, no. 20, 2021, doi: 10.3390/app11209644.
[14] H. Tarazodar, K. Bagherifard, S. Nejatian, H. Parvin, and R. Malekhosseini, “Mitigating concept drift in data streams: an incremental decision tree approach,” Soft Comput., 2024, doi: 10.1007/s00500-024-09921-7.
[15] B. Wang, N. Sun, Z. Wang, and G. Han, “An Adaptive Incremental Learning Algorithm based on Shared Nearest Neighbors in Fault Detection,” in 2021 Computing, Communications and IoT Applications, ComComAp 2021, 2021, pp. 194 – 198. doi: 10.1109/ComComAp53641.2021.9652989.
[16] B. S. Prashanth et al., “Adaptive Buffering Strategies for Incremental Learning Under Concept Drift in Lifestyle Disease Modeling,” IEEE Access, vol. 13, pp. 174001 – 174033, 2025, doi: 10.1109/ACCESS.2025.3616776.
[17] M. Jovanovic and P. Voss, “Towards Incremental Learning in Large Language Models: A Critical Review,” Expert Syst., vol. 42, no. 10, 2025, doi: 10.1111/exsy.70127.
[18] N. Gunasekara, B. Pfahringer, H. M. Gomes, A. Bifet, and Y. S. Koh, “Recurrent Concept Drifts on Data Streams,” in IJCAI International Joint Conference on Artificial Intelligence, 2024, pp. 8029 – 8037. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85204306066&partnerID=40&md5=f7cfbfc96f5583390e8ed672616c1405
[19] H. Mehmood, P. Kostakos, M. Cortes, T. Anagnostopoulos, S. Pirttikangas, and E. Gilman, “Concept drift adaptation techniques in distributed environment for real-world data streams,” Smart Cities, vol. 4, no. 1, pp. 349 – 371, 2021, doi: 10.3390/smartcities4010021.
[20] H. Yu, W. Liu, J. Lu, Y. Wen, X. Luo, and G. Zhang, “Detecting group concept drift from multiple data streams,” Pattern Recognit., vol. 134, 2023, doi: 10.1016/j.patcog.2022.109113.
[21] S. Parasteh and S. Sadaoui, “A Probabilistic Approach for Detecting Real Concept Drift,” in International Conference on Agents and Artificial Intelligence, 2024, pp. 301 – 311. doi: 10.5220/0012378800003636.
[22] A. Gepperth and B. Hammer, “Incremental learning algorithms and applications,” in ESANN 2016 - 24th European Symposium on Artificial Neural Networks, 2016, pp. 357 – 368. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84994180746&partnerID=40&md5=dadbfe8da5c67556866e9f5b6c68e1e5
[23] A. Chefrour, “Incremental supervised learning: algorithms and applications in pattern recognition,” Evol. Intell., vol. 12, no. 2, pp. 97 – 112, 2019, doi: 10.1007/s12065-019-00203-y.
[24] Y. Mittal, U. Kumar, and D. Aggarwal, “Online Incremental Machine Learning Model For Multi-Sensor and Multi-Temporal Satellite Data: A Case Study of Bangalore,” in 2023 IEEE India Geoscience and Remote Sensing Symposium, InGARSS 2023, 2023. doi: 10.1109/InGARSS59135.2023.10490335.
[25] J. P. Barddal, H. M. Gomes, and F. Enembreck, “Analyzing the impact of feature drifts in streaming learning,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9489, pp. 21 – 28, 2015, doi: 10.1007/978-3-319-26532-2_3.
[26] N. L. A. Ghani, I. A. Aziz, and M. Mehat, “Concept Drift Detection on Unlabeled Data Streams: A Systematic Literature Review,” in 2020 IEEE Conference on Big Data and Analytics, ICBDA 2020, 2020, pp. 61 – 65. doi: 10.1109/ICBDA50157.2020.9289802.
[27] H. Hu, M. Kantardzic, and L. Lyu, “Detecting Different Types of Concept Drifts with Ensemble Framework,” in Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018, 2018, pp. 344 – 350. doi: 10.1109/ICMLA.2018.00058.
[28] M. Han, F. Meng, C. Li, R. Zhang, F. He, and J. Ding, “Kolmogorov Inequality Based Drift Detection Methods for Data Stream; [基于 Kolmogorov 不等式的数据流漂移检测方法],” Comput. Eng. Appl., vol. 61, no. 9, pp. 102 – 115, 2025, doi: 10.3778/j.issn.1002-8331.2405-0334.
[29] S. Gamage and U. Premaratne, “Detecting and adapting to concept drift in continually evolving stochastic processes,” in ACM International Conference Proceeding Series, 2017, pp. 109 – 114. doi: 10.1145/3175684.3175723.
[30] S. Greco, B. Vacchetti, D. Apiletti, and T. Cerquitelli, “Unsupervised Concept Drift Detection From Deep Learning Representations in Real-Time,” IEEE Trans. Knowl. Data Eng., vol. 37, no. 10, pp. 6232 – 6245, 2025, doi: 10.1109/TKDE.2025.3593123.
[31] L. Poenaru-Olaru, L. Cruz, A. Van Deursen, and J. S. Rellermeyer, “Are Concept Drift Detectors Reliable Alarming Systems? - A Comparative Study,” in Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022, 2022, pp. 3364 – 3373. doi: 10.1109/BigData55660.2022.10020292.
