Abstract: Vibration recognition plays an important role in realizing structural vibration control and health monitoring and evaluation. Under the background of smart city, high-rise building structures have appealed for higher requirements in their vibrational response recognition. Traditional vibration recognition methods based on time-frequency domain transformation and graph neural network reveal more significant limitations in terms of computational cost and engineering applicability. In this study, a novel vibration recognition method based on deep representation learning of vibration signals is proposed. In present method, self-supervised learning is performed on one-dimensional frequency-domain signals by establishing an autoencoder network to obtain vibration representation information. Then, the vibration sensitive parameters are defined and calculated based on original signals and the reconstructed ones, and the results of which are utilized as feature engineering and fed into the downstream linear classifiers for training, thereby realizing the rapid recognition of vibration. To verify the reliability of the proposed method, the vibration response identification of a super high-rise building under different ambient excitations is conducted for validation. At the same time, in order to prove the superiority of proposed method, the MFCC+CNN vibration recognition method, which is generally considered to be the best at present, is introduced for comparison. The results show that the proposed method is able to save nearly 20 times of computing time, and succeed to improve the overall recognition accuracy from 0.74 to above 0.95.