In recent years, interest in heat recovery systems for building applications has resurged due to concerns about the energy crisis and global climate changes. This review presents current developments in four kinds of heat recovery systems for residential building applications. A extensive investigation into the heat recovery integrated in energy-saving systems of residential buildings is also covered, including passive systems for building components, mechanical/natural ventilation systems, dehumidification systems, and the thermoelectric module (TE) system. Based on this review, key issues have been identified as follows: (1) The combination of heat recovery and energy-efficient systems could be considered as a promising approach to reduce greenhouse gas emissions and make residential buildings meet high performance and comfort requirements. However, real-life evaluation of these systems with economic analysis is insufficient; (2) When heat recovery is applied to mechanical ventilation systems, issues such as pressure leakages and air shortcuts should be addressed; (3) The heat pipe heat recovery system enjoys more potential in being combined with other sustainable technologies such as thermoelectric modules and solar energy systems due to its advantages, which include handy manufacturing and convenient maintenance, a lack of cross contamination, and greater thermal conductance.