The hilly area of Sichuan and Chongqing is an area with high population density and reclamation degree. The terrain is gentle, and the land use is broken. Based on the small-scale and scattered characteristics of its rural settlements, this paper studies the multiple settlements adjacent to each other and related to each other as a whole and puts forward the concept of a “rural settlement unit” from the perspective of natural ecology and rural governance. Through the identification and graphic analysis of rural settlement units in Xiangyi Town, Weiyuan County, Neijiang City, Sichuan Province, and Cizhu Town, Yubei District, Chongqing City, this paper summarizes and extracts the rural settlement mode of “Hills-Ponds-Fields-Forests-Dwellings” in the hilly areas of Sichuan and Chongqing, and analyzes the landscape elements such as hilly ponds, hilly fields, hilly forests, and hilly dwellings. Further contrast the differences of rural settlement landscape between shallow hills and steep hills, and reveal the impact of hilly terrain on rural settlement landscape. Based on the wisdom of traditional rural settlement construction, this paper puts forward opinions and suggestions on the construction of human settlements in contemporary hilly areas.