Firmiana danxiaensis is an endemic plant species which is narrowly distributed in the northern parts of Shaoguan, Guangdong, China. To devise appropriate conservation strategies for this rare species, we examined the population structure and estimated the demographic history of nine natural populations using 18 EST-SSR markers. The results revealed a moderate degree of total genetic diversity (Ht = 0.631) at the species level but high genetic diversity within populations (Hs = 0.546). Strong genetic differentiation existed between two groups of Danxiashan and Nanxiong (FST = 0.150). This current pattern might have arisen from prolonged geographical isolation and human disturbances. STRUCTURE analysis demonstrated that whole population can be divided into two distinct gene pools, closely related by their geographic location and having experienced independent evolutionary routes. The genetic background of Danxiashan was more complicated than that of Nanxiong. Approximate Bayesian Computation (ABC) analysis indicated that divergence time between the two groups could be 100,000 years BP, and their effective population size were 7,290 and 5,550 individuals, respectively. Combining the information of genetic variation with niche modelling, we inferred that F. danxiaensis might have been widely distributed around the Nanling Mountain in the Quaternary period. Populations located in the northern of Nanling Mountain likely became extinct due to climate change. Danxiashan and Nanxiong provided the main refuges for the species during the last ice age. Being a rare species with small population sizes, better ecological investigation of natural populations and in situ conservation and implementation of restoration strategy are important measures to conserve this species.