Accurate and reliable information about the situation in refugee or internally displaced person camps is very important for planning any kind of help like health care, infrastructure, or vaccination campaigns. The number and spatial distribution of single dwellings extracted semi-automatically from very high-resolution (VHR) satellite imagery as an indicator for population estimations can provide such important information. The accuracy of the extracted dwellings can vary quite a lot depending on various factors. To enhance established single dwelling extraction approaches, we have tested the integration of stratified template matching methods in object-based image analysis (OBIA) workflows. A template library for various dwelling types (template samples are taken from ten different sites using 16 satellite images), incorporating the shadow effect of dwellings, was established. Altogether, 18 template classes were created covering typically occurring dwellings and their cast shadows. The created template library aims to be generally applicable in similar conditions. Compared to pre-existing OBIA classifications, the approach could increase the producer’s accuracy by 11.7 percentage points on average and slightly increase the user’s accuracy. These results show that the stratified integration of template matching approaches in OBIA workflows is a possibility to further improve the results of semi-automated dwelling extraction, especially in complex situations.