| 4/2021 - 6 |
Machine Learning Enhanced Entropy-Based Network Anomaly DetectionTIMCENKO, V.   , GAJIN, S. |
View the paper record and citations in  |
| Click to see author's profile in SCOPUS, IEEE Xplore, Web of Science |
Download PDF  (1,765 KB) | Citation | Downloads: 1,291 | Views: 1,321 |
Author keywords
clustering algorithms, data flow computing, entropy, intrusion detection, machine learning
References keywords
detection(22), network(21), security(10), intrusion(10), data(10), anomaly(10), systems(9), learning(8), entropy(8), machine(6)
Blue keywords are present in both the references section and the paper title.
About this article
Date of Publication: 2021-11-30
Volume 21, Issue 4, Year 2021, On page(s): 51 - 60
ISSN: 1582-7445, e-ISSN: 1844-7600
Digital Object Identifier: 10.4316/AECE.2021.04006
Web of Science Accession Number: 000725107100006
SCOPUS ID: 85122239638
Abstract
The advanced development of new technologies and heterogeneous environments relies on the proper processing of large data volumes, and accurate and fast response of real-time applications. Such circumstances provide a fertile ground for the appearance of diverse security concerns, thus challenging the scientific community for building more reliable and efficient Network Anomaly Detection Systems. This research proposes a comprehensive flow-based anomaly detection architecture, which encompasses techniques for entropy-based data processing and machine learning-based attack detection. It encompasses several attack categories and relies on the use of modelled and synthetically generated traffic patterns for Port Scan, Network Scan, DDoS amplification, flood, and dictionary attacks. The entropy-based analysis is used for easier detection of the hidden traffic patterns, as it can capture the behaviour of the biggest contributors, and of a large number of minor appearances in the feature distribution. The unusual traffic is then processed by the use of unsupervised machine learning algorithms. The approach is verified with datasets based on real network traffic, synthetically generated attack traffic instances and botnet traffic. The architecture is an original solution, planned for further real-network application, targeting the possible support for a range of different use cases. |
| References | | | Cited By «-- Click to see who has cited this paper |
| [1] Y. Zhu, "Attack pattern discovery in forensic investigation of network attacks," IEEE J. on Selected Areas in Communications vol. 29, no. 7, pp. 1349-1357, 2011. [CrossRef] [Web of Science Times Cited 9] [SCOPUS Times Cited 16] [2] M. A. Wani, F.A. Bhat, S. Afzal, A.I. Khan, "Introduction to Deep Learning," Advances in Deep Learning. Springer, vol. 57, pp. 1-11, 2020. [CrossRef] [SCOPUS Times Cited 16] [3] M. Antonelli, P. Ducange, B. Lazzerini, F. Marcelloni, "Multi-objective evolutionary design of granular rule-based classifiers," Granular Computing, vol. 1, no. 1, pp. 37-58, 2016. [CrossRef] [Web of Science Times Cited 89] [SCOPUS Times Cited 97] [4] European Union Agency for Network and Information Security - "ENISA Threat Landscape Report 2016: 15 Top Cyber Threats and Trends," 2017. [CrossRef] [5] H. Debar, "Towards a taxonomy of intrusion detection systems," Computer Networks, vol. 31, no. 8, pp. 805-822, 1999. [CrossRef] [Web of Science Times Cited 327] [SCOPUS Times Cited 507] [6] P. Garcia-Teodoro, J. Diaz-Verdejo, G. Macia-Fernandez, E. Vazquez, "Anomaly-based network intrusion detection: techniques, systems and challenges," Computers & Security, vol. 28, no. 1, pp.18-28, 2009. [CrossRef] [Web of Science Times Cited 949] [SCOPUS Times Cited 1307] [7] A. Rehman, S. Tanzila, "Evaluation of artificial intelligent techniques to secure information in enterprises," Artificial Intelligence Review, vol. 42, no. 4, pp. 1029-1044, 2014. [CrossRef] [Web of Science Times Cited 30] [SCOPUS Times Cited 48] [8] H. Bostani, S. Sheikhan, "Hybrid of anomaly-based and specification-based IDS for internet of things using unsupervised OPF based on MapReduce approach," Computer Communications, vol. 98, pp. 52-71, 2017. [CrossRef] [Web of Science Times Cited 130] [SCOPUS Times Cited 184] [9] N. Moustafa, J. Hu, J. Slay, "A holistic review of network anomaly detection systems: a comprehensive survey," J. of Network and Computer Applications, vol. 128, pp. 33-55, 2019. [CrossRef] [Web of Science Times Cited 148] [SCOPUS Times Cited 224] [10] G. Peters, R. Weber, "DCC: A framework for dynamic granular clustering," Granular Computing, vol. 1, no. 1, pp. 1-11, 2016. [CrossRef] [Web of Science Times Cited 119] [SCOPUS Times Cited 123] [11] N. Shone, T. N. Ngoc, V.D. Phai, Q. Shi, "A deep learning approach to network intrusion detection," IEEE Trans. Emerging Topics in Computational Intelligence, vol. 2, no. 1, pp. 41-50, 2018. [CrossRef] [Web of Science Times Cited 679] [SCOPUS Times Cited 1011] [12] M. Tavallaee, E. Bagheri, W. Lu, A. Ghorbani, "A detailed analysis of the KDD CUP 99 data set," IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, 2009, pp. 1-6, [CrossRef] [SCOPUS Times Cited 3182] [13] L. Dhanabal, S. P. Shantharajah, "A study on NSL-KDD dataset for intrusion detection system based on classification algorithms," Int. J. of Advanced Research in Computer and Communication Engineering, vol. 4, no. 6, pp. 446-452, 2015. [CrossRef] [14] M. Ring, S. Wunderlich, D. Grudl, D. Landes, A. Hotho, "A toolset for intrusion and insider threat detection," Data Analytics and Decision Support for Cybersecurity: Trends, Methodologies and Applications, Springer, pp. 3-31, 2017. [CrossRef] [Web of Science Times Cited 8] [15] M. Ring, S. Wunderlich, D. Grudl, D. Landes, A. Hotho, "A survey of network-based intrusion detection data sets," Computers & Security, vol. 86, pp. 147-167, 2019. [CrossRef] [Web of Science Times Cited 313] [SCOPUS Times Cited 445] [16] A. Sperotto, R. Sadre, F. Van Vliet, A. Pras, "A labeled data set for flow-based intrusion detection," IP Operations & Management Springer, vol. 5843, pp. 39-50, 2009. [CrossRef] [SCOPUS Times Cited 119] [17] P. Winter, E. Hermann, M. Zeilinger, "Inductive intrusion detection in flow-based network data using one-class support vector machines," in 4th IFIP Int. Conf. on New Technologies, Mobility and Security, pp. 1-5, 2011. [CrossRef] [SCOPUS Times Cited 78] [18] N. Moustafa, J. Slay, "UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set)," in Proc. MilCIS, IEEE, pp. 1-6, 2015. [CrossRef] [SCOPUS Times Cited 1975] [19] G. Nychis et al, "An empirical evaluation of entropy-based traffic anomaly detection," 8th ACM SIGCOMM Conf. Internet Measurement, pp. 151, 2008. [CrossRef] [SCOPUS Times Cited 277] [20] B. Li et al, "A survey of network flow applications," J. of Network and Computer Applications, vol. 36, no. 2, pp. 567-581, 2013. [CrossRef] [Web of Science Times Cited 151] [SCOPUS Times Cited 180] [21] P. Berezinski, M. Szpyrka, B. Jasiul, M. Mazur, "Network anomaly detection using parameterized entropy," Computer Information Systems and Industrial Management, Springer, vol. 8838, pp. 465-478, 2014. [CrossRef] [SCOPUS Times Cited 22] [22] K. Xu, Z.L. Zhang, S. Bhattacharyya, "Internet traffic behavior profiling for network security monitoring," IEEE/ACM Trans. Networking, vol. 16, no. 6, pp. 1241-1252, 2008. [CrossRef] [Web of Science Times Cited 63] [SCOPUS Times Cited 88] [23] B. Agarwal, N. Mittal, "Hybrid approach for detection of anomaly network traffic using data mining techniques," Procedia Technology, vol. 6, pp. 996-1003, 2012. [CrossRef] [Web of Science Times Cited 42] [24] C. Zhua, Z. Wang, "Entropy-based matrix learning machine for imbalanced data sets," Pattern Recognition Letters, vol. 88, pp. 72-80, 2010. [CrossRef] [Web of Science Times Cited 32] [SCOPUS Times Cited 43] [25] Y. Zhang et al, "Comparison of machine learning methods for stationary wavelet entropy-based multiple sclerosis detection: decision tree, k-nearest neighbors, and support vector machine," Simulation, vol. 92, no. 9, pp. 861-871, 2016. [CrossRef] [Web of Science Times Cited 127] [SCOPUS Times Cited 141] [26] Y. Wang et al, "Internet traffic classification using constrained clustering," IEEE Trans. Parallel and Distributed Systems, vol. 25, no. 11, pp. 2932-2943, 2013. [CrossRef] [Web of Science Times Cited 77] [SCOPUS Times Cited 85] [27] T. Mitchell. Machine Learning. McGraw Hill Boston, 1997. [28] S. B. Kotsiantis, I. Zaharakis, P. Pintelas, "Supervised machine learning: A review of classification techniques," Emerging Artificial Intelligence Applications in Computer Engineering, vol. 160, pp. 3-24. OS Press, 2007. [29] B. Claise, G. Sadasivan, V. Valluri, M. Djernaes, "Cisco Systems NetFlow Services Export v. 9," RFC 3954, 2004. [30] J. A. Ibrahim, S. Gajin, "Entropy-based network traffic anomaly classification method resilient to deception," Computer Science and Information Systems, pp. 45-45, 2021. [CrossRef] [Web of Science Times Cited 3] [SCOPUS Times Cited 6] [31] A. Webster, M. Gratian, R. Eckenrod, D. Patel, M. Cukier, "An improved method for anomaly-based network scan detection," Security and Privacy in Communication Systems, Springer, vol. 164, pp. 385-400, 2015. [CrossRef] [SCOPUS Times Cited 4] [32] S. Andropov, A. Guirik, M. Budko, M. Budko, "Network anomaly detection using artificial neural networks," 20th Conf. Open Innovations Association, pp. 26-31, 2017. [CrossRef] [SCOPUS Times Cited 18] [33] J. Mirkovic, P. Reiher, "A taxonomy of DDoS attack and DDoS defense mechanisms," ACM SIGCOMM Computer Communication Review, vol. 34, no. 2, pp. 39-53, 2004. [CrossRef] [Web of Science Times Cited 842] [SCOPUS Times Cited 1294] [34] J. Nazario, "DDoS attack evolution," Network Security, vol. 7, pp. 7-10, 2008. [CrossRef] [SCOPUS Times Cited 40] [35] C. Douligeris, A. Mitrokotsa, "DDoS attacks and defense mechanisms: classification and state-of-the-art," Computer Networks, vol. 44, no. 5, pp. 643-666, 2004. [CrossRef] [Web of Science Times Cited 332] [SCOPUS Times Cited 511] [36] H. Wang, D. Zhang, K. G. Shin, "Detecting SYN flooding attacks," Proc. 21st Joint Conf. IEEE Computer and Communications Societies, vol. 3, pp. 1530-1539, 2002. [CrossRef] [37] G. Kambourakis, T. Moschos, D. Geneiatakis, S. Gritzalis, "Detecting DNS amplification attacks," Critical Information Infrastructures Security," Lecture Notes in Computer Science, vol. 5141, pp. 185-196, 2008. Springer. [CrossRef] [SCOPUS Times Cited 72] [38] J. Vykopal, T. Plesnik, P. Minarik, "Network-based dictionary attack detection," Int. Conf. Future Networks, pp. 23-27, 2009. [CrossRef] [Web of Science Times Cited 14] [SCOPUS Times Cited 23] [39] A. Wagner, B. Plattner, "Entropy based worm and anomaly detection in fast IP networks," 14th IEEE Int. Workshops on Enabling Technologies Infrastructure for Collaborative Enterprise, pp. 172-177, 2005. [CrossRef] [Web of Science Times Cited 109] [SCOPUS Times Cited 199] [40] J. M. Amigo, G. B. Samuel, S. Hernandez, "A brief review of generalized entropies," Entropy, vol. 20, no. 11, pp. 813, 2018. [CrossRef] [Web of Science Times Cited 111] [SCOPUS Times Cited 116] [41] L. Lima, F. M. Assis, C. P. de Souza, "A comparative study of use of Shannon, Renyi and Tsallis entropy for attribute selecting in network intrusion detection," Int. Workshop on Measurements and Networking, pp. 77-82, 2011. [CrossRef] [SCOPUS Times Cited 16] [42] A. J. Lawrance, P.A.W. Lewis, "An exponential moving-average sequence and point process (EMA1)," J. of Applied Probability, vol. 14, no. 1, pp. 98-113, 1977. [CrossRef] [Web of Science Times Cited 75] [43] C. E. Shannon, "A mathematical theory of communication," Bell System Technical J., vol. 27, no. 3, pp. 379-423, 1948. [CrossRef] [SCOPUS Times Cited 32492] [44] P.D. Bojovic, I. Basicevic, S. Ocovaj, M. Popovic, "A practical approach to detection of distributed denial-of-service attacks using a hybrid detection method," Computers & Electrical Engineering, vol. 73, pp. 84-96, 2018. [CrossRef] [Web of Science Times Cited 28] [SCOPUS Times Cited 44] [45] T. K. Moon, "The expectation-maximization algorithm," IEEE Signal Processing Magazine, vol. 13, no. 6, pp. 47-60, 1996. [CrossRef] [Web of Science Times Cited 1870] [SCOPUS Times Cited 2284] [46] B. Xue, M. Zhang, W. N. Browne, X. Yao, "A survey on evolutionary computation approaches to feature selection," IEEE Trans. Evolutionary Computation, vol. 20, no. 4, pp. 606-626, 2016. [CrossRef] [Web of Science Times Cited 1034] [SCOPUS Times Cited 1261] [47] S. Garcia, M. Grill, J. Stiborek, A. Zunino, "An empirical comparison of botnet detection methods," Computers and Security J., vol. 45, pp. 100-123, 2014. [CrossRef] [Web of Science Times Cited 415] [SCOPUS Times Cited 589] [48] S. Garcia, "Malware capture facility project," CVUT University. Dataset CTU-Malware-Capture-Botnet-1, 2013. [49] F. Azuaje, I.H. Witten, E. Frank, "Data mining: practical machine learning tools and techniques," BioMedical Engineering OnLine, vol. 5, no. 1, pp. 51, 2006. [CrossRef]1 Web of Science® Citations for all references: 8,126 TCR SCOPUS® Citations for all references: 49,137 TCR Web of Science® Average Citations per reference: 163 ACR SCOPUS® Average Citations per reference: 983 ACR TCR = Total Citations for References / ACR = Average Citations per Reference We introduced in 2010 - for the first time in scientific publishing, the term "References Weight", as a quantitative indication of the quality ... Read more Citations for references updated on 2024-04-26 16:13 in 262 seconds. Note1: Web of Science® is a registered trademark of Clarivate Analytics. Note2: SCOPUS® is a registered trademark of Elsevier B.V. Disclaimer: All queries to the respective databases were made by using the DOI record of every reference (where available). Due to technical problems beyond our control, the information is not always accurate. Please use the CrossRef link to visit the respective publisher site. |
Copyright ©2001-2024
Faculty of Electrical Engineering and Computer Science
Stefan cel Mare University of Suceava, Romania
All rights reserved: Advances in Electrical and Computer Engineering is a registered trademark of the Stefan cel Mare University of Suceava. No part of this publication may be reproduced, stored in a retrieval system, photocopied, recorded or archived, without the written permission from the Editor. When authors submit their papers for publication, they agree that the copyright for their article be transferred to the Faculty of Electrical Engineering and Computer Science, Stefan cel Mare University of Suceava, Romania, if and only if the articles are accepted for publication. The copyright covers the exclusive rights to reproduce and distribute the article, including reprints and translations.
Permission for other use: The copyright owner's consent does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific written permission must be obtained from the Editor for such copying. Direct linking to files hosted on this website is strictly prohibited.
Disclaimer: Whilst every effort is made by the publishers and editorial board to see that no inaccurate or misleading data, opinions or statements appear in this journal, they wish to make it clear that all information and opinions formulated in the articles, as well as linguistic accuracy, are the sole responsibility of the author.
Faculty of Electrical Engineering and Computer Science
Stefan cel Mare University of Suceava, Romania
All rights reserved: Advances in Electrical and Computer Engineering is a registered trademark of the Stefan cel Mare University of Suceava. No part of this publication may be reproduced, stored in a retrieval system, photocopied, recorded or archived, without the written permission from the Editor. When authors submit their papers for publication, they agree that the copyright for their article be transferred to the Faculty of Electrical Engineering and Computer Science, Stefan cel Mare University of Suceava, Romania, if and only if the articles are accepted for publication. The copyright covers the exclusive rights to reproduce and distribute the article, including reprints and translations.
Permission for other use: The copyright owner's consent does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific written permission must be obtained from the Editor for such copying. Direct linking to files hosted on this website is strictly prohibited.
Disclaimer: Whilst every effort is made by the publishers and editorial board to see that no inaccurate or misleading data, opinions or statements appear in this journal, they wish to make it clear that all information and opinions formulated in the articles, as well as linguistic accuracy, are the sole responsibility of the author.
