Total factor productivity (TFP) is key for sustainable agriculture, the higher growth of TFP, the more output of agriculture sector. TFP index is often employed to measure growth in the total factor productivity and the source attributed to its growth, which theoretically is defined as the ratio of an aggregate output index to an aggregate input index. In previous literature, the agricultural TFP index has been widely used in different spatial scales of China, such as province regions scale and county regions scale, however, the analysis of agricultural TFP index in Chongqing is scarce. In addition, although the analysis of agricultural TFP index under constraint of resources and environment have been conducted in many studies, our knowledge about the impact of climate change on agricultural TFP growth is still limited, correspondingly the adaptation measures to climate change are also rarely discussed in previous studies. The aim of this study is to detect the spatial and temporal changes of total factor productivity in Chongqing agriculture, and explore the influence of climate change on TFP growth in Chongqing agriculture. The results show that: 1) Under the constraints of climatic factors, the value of TFP growth is relative lower but with more significant spatial heterogeneity, and the change of TFP growth can reflect the influence of climate change and technical change on agricultural production. 2) The drivers of TFP growth come from technical improvement, but negative factor of TFP growth come from mix efficiency change. 3) The high value of TFP growth focused on the western region of Chongqing, however, the lower value focused on the main city of Chongqing. 4) The extreme drought in 2006 led to a lower TFP growth of Chongqing agriculture, so agricultural adaptation measures to climate change should be taken for different regions of Chongqing. Our results can help policy makers identify where agricultural productivity growth is strongest and weakest, and thereby assist in illuminating specific policies and practices on supply-side reform of agricultural structure.