High spatial resolution remote sensing (HSRRS) images classification and identification is an important technology to acquire land surface information for land resource management, geographical situation monitoring, and global climate change. As the hottest deep learning method, convolutional neural network (CNN) has been successfully applied in HSRRS image classification and identification due to its powerful information extraction capability. However, adversarial perturbations caused by radiation transfer process or artificial or other unpredictable disturbances often deteriorate the stability of CNN. Under this background, we propose a robust architecture for adversarial attack and detection to classify and identify HSRRS images. First of all, two white-box attacks [i.e., large Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) and fast gradient sign method (FGSM)] are adopted respectively to generate adversarial images to confuse the model, and to assess the robustness of the HSRRS image classifier. Second, adversarial detection models based on support vector machine (SVM) with single or fused two level features are proposed to improve the detection accuracy. The features extracted from the testing CNN full connected layers contain adversarial perturbations and real information, from which SVM classifier and discriminate the real and the adversarial images. The adversarial attack model is evaluated in terms of overall accuracy (OA) and kappa coefficient (kc). The simulation results show that the OA decreases from 96.4% to 44.4% and 33.3% for L-BFGS and FGSM attacked classifier model, respectively. The adversarial detection is evaluated via OA, detection probability P-D, false alarm probability P-FA, and miss probability P-M. The simulation results indicate that the fused model with two different level features based on SVM can obtain the best OA (94.5%), P-D (0.933), P-FA (0.040), and P-M (0.067) among the detectors if the classifier is attacked by the FGSM. Meanwhile, when facing the L-BFGS attack, the fused model presents similar performance if the best single level features are utilized.