To explore the distribution patterns of alpine glaciers and their response mechanisms to geographical location, topographic, and climatic factors, we used the Manas River Basin in Xinjiang as the study area. We selected TM images, DEM, and meteorological station data as our data sources, and with the aid of remote sensing (RS) and GIS technology, we extracted the glacial boundary information and used a linear regression model to estimate the temperature and precipitation data of the studied region and to carry out grid calculation. Based on the DEM data, we extracted topographic information, such as the slope and aspect of the mountainous region. After overlaying this information onto the studied glacial region, we obtained 213 uniformly distributed glacier sample points and their attribute data, such as geolocation (latitude and longitude), topographic features (elevation, slope, and aspect), Normalized Differential Snow Index (NDSI), atmospheric temperature and precipitation. Finally, we constructed a glacier-topography-climate RS inversion model using the partial least squares (PLS) and tested its effectiveness and adaptability. The results showed that the model could explain more than 78.2% of the variance of the dependent variables and that its overall ability to estimate the glacial distribution patterns of the studied region reached 76.9%. The effectiveness testing highlighted the efficacy of the model and its ability to successfully overcome the interference caused by the multiple correlations of the independent variables. Simultaneously, we found that PLS method had better analytical ability in relation to glacier distribution pattern and its influencing factors. Analysis of the response mechanism of the glacial distribution patterns to the influencing factors in the study region showed that climate change was the most important factor affecting the dynamic changes in the glacial distribution patterns. Topographic factors played a decisive role in glacier development and accumulation, whereas geolocation was a relatively small contributor. Moreover, to improve the glacial distribution modeling, in future research, other factors that significantly affect the distribution patterns should be introduced into the model, e.g., wind direction and strength, water vapor pressure, relative humidity, and the properties of the underlying surface of the studied mountainous region. This study will be a theoretical reference for research on regional climate and hydrology and provide significant guidance for rational utilization of water resources in arid areas.