Deep Reinforcement Learning based Handoff Algorithm in End-to-End Network Slicing Enabling HetNets

End-to-end network slicing, as a key technology in 5G and B5G mobile communication systems, is to enable traditional wireless networks to support different services in vertical industries. In a heterogeneous cellular network (HetNets) with network slicing functions, due to the dense deployment of base stations (BSs) and the mobility of user equipments (UEs), dynamically switching network slices (NSs) is necessary for better system performance. This paper models the handoff problem of end-to-end NS as a Markov decision process (MDP) maximizing the utility related to the UE’s profit of being served, the handoff cost and the outage penalty. Both the states of the radio access network resources and the core network resources of each end-to-end NS are considered. The deep reinforcement learning (DRL) is adopted as the solution, and a double deep Q network (DQN) based NS handoff algorithm is designed. Numerical results confirm the convergence of the DQN used to make handoff decisions and show that compared with typical handoff algorithms, the algorithm we proposed performs the best from the aspect of the cumulative reward designed in this paper.