DBC-MulBiLSTM: A DistilBERT-CNN Feature Fusion Framework enhanced by multi-head self-attention and BiLSTM for smart contract vulnerability detection

Abstract

With the burgeoning of blockchain technology, particularly the Ethereum platform, smart contracts, serving as the core technology of blockchain, have demonstrated immense potential in numerous fields. However, vulnerabilities in smart contracts have also become targets for cyberattacks, potentially leading to significant economic losses. This study introduces a DBC-MulBiLSTM framework designed for the detection of vulnerabilities in smart contracts. The framework first utilizes the lightweight pre-trained model DistilBERT to extract contextual features from smart contracts, while simultaneously utilizing Convolutional Neural Networks (CNN) to identify local features. Through feature fusion, a multi-dimensional feature representation is formed to improve the model's capabilities to recognize complex vulnerability patterns. Furthermore, the framework incorporates a multi-head self-attention mechanism within the BiLSTM architecture, thereby establishing the MulBiLSTM training framework. This design enables the simultaneous capture of long-range dependencies throughout the entire dataset, enhancing the model's ability to represent intricate dependencies and contextual information effectively. Experimental results demonstrate that DBC-MulBiLSTM exhibits substantial efficacy in the detection of vulnerabilities within smart contracts, achieving an F1 score of 95.44%, an accuracy rate of 96.57%, and a recall of 95.36%. For various vulnerability types, the model consistently achieves accuracy and F1-scores over 96%, and recall rates above 95%, showcasing efficient and accurate smart contract vulnerability detection capabilities.