Digital Forensics and Automated Incident Response Framework Leveraging Big Data Analytics and Real Time Network Traffic Profiling in Heterogeneous Cyber Environments
Keywords:
Digital forensics, Incident response, Big data, Network traffic, Real time analysisAbstract
The increasing complexity and scale of modern cybersecurity threats necessitate the development of advanced systems capable of efficiently detecting, analyzing, and mitigating incidents in real time. This paper proposes an automated framework for digital forensics and incident response that leverages big data analytics and real time network traffic profiling. The framework integrates cutting-edge technologies, including Apache Spark for real time data processing and Hadoop for scalable data storage, combined with machine learning models like LSTM and Autoencoders to detect anomalies and threats in network traffic. By automating the process of incident detection and response, this framework significantly reduces the time required to identify threats and improves the accuracy of forensic evidence correlation across heterogeneous network environments. The study highlights the advantages of using machine learning models and big data tools to address the limitations of traditional manual and semi-automated systems, which often struggle to keep pace with large-scale data generation. Testing results demonstrate that the proposed framework can handle large data volumes efficiently, providing real time, actionable insights with significantly reduced response times. Additionally, the framework improves forensic analysis by enabling the correlation of evidence from different devices and protocols, making it more effective than traditional methods in identifying the root cause of security incidents. However, challenges related to data heterogeneity, scalability, and system integration were encountered during testing. The proposed framework holds promise for significantly enhancing the efficiency and effectiveness of cybersecurity operations, with future work focusing on further integration of advanced AI techniques and machine learning models for dynamic and adaptive incident response.
References
[1] I. Homem, T. Kanter, and R. Rahmani, “Improving distributed forensics and incident response in loosely controlled networked environments,” Int. J. Secur. its Appl., vol. 10, no. 1, pp. 385 – 414, 2016, doi: 10.14257/ijsia.2016.10.1.35.
[2] R. A. Hansen et al., “File Toolkit for Selective Analysis Reconstruction (FileTSAR) for Large-Scale Networks,” in Proceedings - 2018 IEEE International Conference on Big Data, Big Data 2018, 2018, pp. 3059 – 3065. doi: 10.1109/BigData.2018.8621914.
[3] N. Kumari, T. Sharma, A. K. Gupta, and G. Dua, “Taxonomy of Technical Challenges in Digital Forensics,” in Proceedings of the IEEE International Conference Image Information Processing, 2023, pp. 454 – 458. doi: 10.1109/ICIIP61524.2023.10537638.
[4] R. A. Awad, S. Beztchi, J. M. Smith, B. Lyles, and S. Prowell, “Tools, techniques, and methodologies: A survey of digital forensics for SCADA systems,” in ACM International Conference Proceeding Series, 2018, pp. 1 – 8. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85060632270&partnerID=40&md5=0c79dd9044387136a62fddbb6e818b30
[5] N. Raza, “Challenges to network forensics in cloud computing,” in Proceedings - 2015 Conference on Information Assurance and Cyber Security, CIACS 2015, 2016, pp. 22 – 29. doi: 10.1109/CIACS.2015.7395562.
[6] V. Machaka and T. Balan, “Investigating Proactive Digital Forensics Leveraging Adversary Emulation,” Appl. Sci., vol. 12, no. 18, 2022, doi: 10.3390/app12189077.
[7] T. Janarthanan, M. Bagheri, and S. Zargari, “IoT Forensics: An Overview of the Current Issues and Challenges,” Adv. Sci. Technol. Secur. Appl., pp. 223 – 254, 2021, doi: 10.1007/978-3-030-60425-7_10.
[8] N. Nelufule, P. Senamela, and P. Moloi, “Digital Forensics Investigations on Evolving Digital Ecosystems and Big Data Sharing: A Survey of Challenges and Potential Opportunities,” IST-Africa, no. 2025, 2025, doi: 10.23919/IST-Africa67297.2025.11060495.
[9] A. Sharma and A. Chaudhary, “Automated Incident Response System for Cybersecurity Threat Mitigation,” Lect. Notes Networks Syst., vol. 1653 LNNS, pp. 536 – 545, 2026, doi: 10.1007/978-3-032-06694-7_50.
[10] A. Tripathi, S. Shrivastava, and K. Praveen, “Enhancing Digital Forensic Readiness: Automated Detection of Missing and Null Log Values,” Lect. Notes Electr. Eng., vol. 1219 LNEE, pp. 511 – 520, 2025, doi: 10.1007/978-981-97-4540-1_37.
[11] C. Mpungu, C. George, and G. Mapp, “Digital Forensics Readiness in Big Data Networks: A Novel Framework and Incident Response Script for Linux–Hadoop Environments,” Appl. Syst. Innov., vol. 7, no. 5, 2024, doi: 10.3390/asi7050090.
[12] D. Alharthi, “Cloud Incident Response Framework and AI-Based Forensics Using Reinforcement Learning and Graph Neural Networks,” in 2024 IEEE 15th Annual Information Technology, Electronics and Mobile Communication Conference, IEMCON 2024, 2024, pp. 164 – 170. doi: 10.1109/IEMCON62851.2024.11093338.
[13] H.-C. Yang, I.-L. Lin, and Y.-H. Chao, “Enhancing Traditional Reactive Digital Forensics to a Proactive Digital Forensics Standard Operating Procedure (P-DEFSOP): A Case Study of DEFSOP and ISO 27035,” Appl. Sci., vol. 15, no. 18, 2025, doi: 10.3390/app15189922.
[14] E. E.-D. Hemdan and D. H. Manjaiah, Digital Investigation of Cybercrimes Based on Big Data Analytics Using Deep Learning. 2019. doi: 10.4018/978-1-7998-0414-7.ch034.
[15] S. S. M. Rahman and C. L’Abbe, “Digital forensics and incident response recommendations for an enterprise to build resiliency against cyber crimes,” in Proceedings of the 30th International Conference on Computers and Their Applications, CATA 2015, 2015, pp. 437 – 442. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84925949829&partnerID=40&md5=9cab2a4087d2d831e2c18ff9defca57a
[16] A. Patil, S. Banerjee, D. Jadhav, and G. Borkar, Roadmap of Digital Forensics Investigation Process with Discovery of Tools. 2021. doi: 10.1002/9781119795667.ch11.
[17] S. Li and Y. Liu, “Human-centric Artificial Intelligence enabled Digital Images and Videos Forensic Triage,” in Proceedings - 2023 Human-Centered Cognitive Systems, HCCS 2023, 2023. doi: 10.1109/HCCS59561.2023.10452651.
[18] R. Montasari, R. Hill, V. Carpenter, and A. Hosseinian-Far, “The standardised digital forensic investigation process model (SDFIPM),” Adv. Sci. Technol. Secur. Appl., pp. 169 – 209, 2019, doi: 10.1007/978-3-030-11289-9_8.
[19] E. Çakir and A. Ç. Tolga, “A Review of Artificial Intelligence′s Impact on Cybersecurity in the Big Data Era,” Lect. Notes Comput. Sci., vol. 15886 LNCS, pp. 182 – 192, 2026, doi: 10.1007/978-3-031-97576-9_12.
[20] M. A. Ameedeen, R. A. Hamid, T. H. H. Aldhyani, L. A. K. M. Al-Nassr, S. O. Olatunji, and P. Subramanian, “A Framework for Automated Big Data Analytics in Cybersecurity Threat Detection,” Mesopotamian J. Big Data, vol. 2024, pp. 175 – 184, 2024, doi: 10.58496/MJBD/2024/012.
[21] U. R. Chityala, A. H. Shnain, M. Govindaraj, P. Johri, T. Kuppuraj, and N. L. Devi, “Big Data for Enhancing Cybersecurity in Enterprise Environments Proactive Threat Detection and Prevention,” in 2025 International Conference on Automation and Computation, AUTOCOM 2025, 2025, pp. 1396 – 1401. doi: 10.1109/AUTOCOM64127.2025.10957069.
[22] A. Naseer and A. M. Siddiqui, “The Effect of Big Data Analytics in Enhancing Agility in Cybersecurity Incident Response,” in 2022 16th International Conference on Open Source Systems and Technologies, ICOSST 2022 - Proceedings, 2022. doi: 10.1109/ICOSST57195.2022.10016853.
[23] S. Qiao, Q. Guo, M. Wang, H. Zhu, J. J. P. C. Rodrigues, and Z. Lyu, “Advances in network flow watermarking: A survey,” Comput. Secur., vol. 159, 2025, doi: 10.1016/j.cose.2025.104653.
[24] R. V Umaselvi and T. R. Nisha Dayana, “A Hybrid Technique for Detecting Cyber Threats Through Network Traffic Analysis,” in Proceedings - 2025 5th International Conference on Expert Clouds and Applications, ICOECA 2025, 2025, pp. 562 – 566. doi: 10.1109/ICOECA66273.2025.00102.
[25] S. Kaloria, R. K. Saxena, and D. Bairwa, “INTELLIGENT NETWORK TRAFFIC ANALYSIS: LEVERAGING MACHINE LEARNING FOR ENHANCED CYBERSECURITY,” IET Conf. Proc., vol. 2024, no. 38, pp. 96 – 100, 2024, doi: 10.1049/icp.2025.0777.
[26] E. El-Din Hemdan and D. H. Manjaiah, Digital investigation of cybercrimes based on big data analytics using deep learning. IGI Global, 2017. doi: 10.4018/978-1-5225-3015-2.ch005.
[27] P. Rathore and K. Kolhe, “Integrating Automation and Orchestration in Security Incident Handling: A Review of SOAR Frameworks and Platforms,” Mech. Mach. Sci., vol. 185, pp. 529 – 551, 2026, doi: 10.1007/978-3-031-95963-9_38.
[28] Y. A. Farrukh, S. Wali, I. Khan, and N. D. Bastian, “XG-NID: Dual-modality network intrusion detection using a heterogeneous graph neural network and large language model,” Expert Syst. Appl., vol. 287, 2025, doi: 10.1016/j.eswa.2025.128089.
[29] A. L. Lois, C. K. K. Reddy, and M. Singh, Artificial Intelligence in Cybersecurity: Fundamentals, Challenges, and Opportunities. 2025. doi: 10.1201/9781003631507-1.
[30] A. Dehghantanha, R. M. Parizi, and G. Epiphaniou, “AutonomousCyber’24: Workshop on Autonomous Cybersecurity,” in CCS 2024 - Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security, 2024, pp. 4911 – 4913. doi: 10.1145/3658644.3701044.
[31] D. Goyal, Y. Gandhi, D. Dongre, Shailesh, G. P. Bhagat, and R. Pawar, “Decision Systems for Adaptive Cybersecurity Incident Response,” Smart Innov. Syst. Technol., vol. 422, pp. 543 – 563, 2025, doi: 10.1007/978-981-96-0147-9_45.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Cyber Security and Network Management
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
