Under the background of global climate change, the response of crop production to meteorological conditions and climate variability has attracted widespread attention. Based on the official meteorological and agricultural production data issued by Hunan Province from 1980 to 2016, we analyzed the change characteristics of local climate factors and crop yields, and the influences of climate factors on crop output in high, medium and low production areas with a variety of statistical methods. The results show that: (1) In 1980-2016, annual average temperature in Hunan Province increased significantly at the rate of 0.39/10 a. Annual average precipitation and sunshine hour decreased but not significantly. There were sudden changes in meteorological factors generally. (2) Per unit yields of four major crops all showed upward trend, with the increase rate ranging from high to low: maize(0.122 t/hm~2/a)>soybeans(0.040 t/hm~2/a)>wheat (0.039 t/hm~2/a)> rice(0.028 t/hm~2 /a). Rice had 0 climate good year; wheat 9 climate good years; maize 19 climate good years; and soybeans 11 climate good years. Rice had 1 climate lean year; wheat 5 climate lean years; maize 2 climate lean years; and soybeans 4 climate lean years. (3) Annual average temperature had a significant negative influence on the climate yield of rice in medium production areas. Annual average precipitation had a significant negative influence on the climate yield of rice and wheat in low production areas. Annual average sunshine hour had a significant positive influence on the climate yield of maize in medium production areas, and rice, wheat and maize in low production areas. Our study suggested that the cropping structure and cropping system could be further optimized in high production area. Meanwhile, improved breeds of rice and wheat resistant to high temperature and waterlogging could be introduced and cultivated in low/medium production areas. Their planting period could also be adjusted appropriately.