Urban sprawl is now paving out the land, which leads to the contradiction between land and man. This problem is particularly prominent in cities where construction land is rugged. Microgreen spaces can reduce unserved zones under limited construction areas and provide efficient recreational services. Shiyan City, a significant mountainous city on the middle line of the south-to-north water diversion project, is restricted by various factors in planning its green spaces. This research establishes evaluation methods for microgreen spaces and takes the block as the research unit to measure the space efficiency of microgreen spaces in Shiyan City. Then based on the matching results of space efficiency and recreational demand, the demand points are identified. The potential microgreen space construction points have been selected from the fragmented green spaces and the industries that will suppress from the second industry to the third industry. The k-means and LA location-allocation models are comprehensively operated to optimize the site selection and layout of small and microgreen space for the current “low efficiency-high demand” plot to achieve maximum coverage. The research results show that: (1) Under the current limited range of miniature and microgreen spaces in Shiyan City, the number of low-efficiency and low-efficiency plots accounted for 19.1% and 20.2% of the total, respectively, and a large number of service blind spots caused the supply and demand of green space. (2) 63.45% of the plots are in a contradiction between low efficiency and high demand, but residents’ recreational needs rarely are met. Through model optimization, the study added 113 small and microgreen spaces based on maintaining the existing green space layout to achieve maximum coverage of urban demand points. Comparing the results before and after optimization shows that: (1) In terms of space efficiency, the low space efficiency plots decreased from 46 to 29, the high space efficiency plots increased from 59 to 138, and the high space efficiency plots increased from 31 plots. (2) Regarding space efficiency and recreational demand matching, the proportion of low efficiency-high demand land plots has been reduced from 63.45% to 17.73%, reaching the preset target. The optimized layout improves residents’ enjoyment of small and microgreen spaces opportunities for recreational services.