Abstract: The displacement monitoring of retaining structure can provide an analysis basis for evaluating its working state, disease detection and disaster warning. In order to solve the shortcomings of traditional monitoring methods, such as low efficiency, high cost and high-risk factor, proposed is a UAV-based 3D reconstruction of retaining structure and full-field displacement monitoring method. A UAV is used to capture multi-view images of the surface of the retaining structure, and Colmap algorithm is employed to calculate the pose corresponding to each photo and 3D sparse point cloud. The Patchmatchnet algorithm based on deep learning is utilized to reconstruct the 3D dense point cloud. M3C2 algorithm is adopted to compare the point clouds of two phases in different periods, and the full-field displacement data and displacement cloud image of the retaining structure is obtained. In order to verify the accuracy of this method, a physics-based graphical model (PBGM) was used in the verification experiment. The Abaqus finite element model of the target structure was established, and the load was applied in the finite element model to calculate the model response. The deformed model and undeformed model were exported to Blender virtual space, and the UAV flight path was set after texture mapping. The full-field displacement calculated by this method was compared with that calculated by finite element method under three working conditions, which verified the method in PBGM model test. The errors of model length, of width and, of displacement monitoring direction are less than 1 cm. In the aspect of engineering application, the method is applied to a frame composite anchor soil nail wall in Lanzhou, and its full-field displacement is successfully monitored, which proves that the method is suitable for the displacement monitoring of retaining structures that are difficult for personnel to reach, to steep and dangerous, and can provide a basis for the safety and stability evaluation of retaining structures.