Semantic segmentation is widely used in medical image segmentation, unmanned driving, remote sensing image segmentation and other computer vision tasks. In order to solve the problem of deploying embedded platforms with limited computing power and hardware storage, a lightweight semantic segmentation model is proposed by considering three aspects of network parameters, calculation and performance. The model takes the lightweight network MobileNetV2 as the backbone, depthwise separable convolution is applied to compress the model, which is divided into two paths of high and low semantic features for derivation. High-semantic features can obtain accurate contextual information through the dual attention pyramid pooling module. Low-semantic features can obtain clearer segmentation boundary by multi-scale feature stitching and high semantic information transmission. Finally, high and low semantic features are fused to obtain the segmentation results. In the experiments on PASCAL VOC 2012 dataset, compared with the mainstream network model, the number of network parameters of model is 2.31×106, which is only 4.9% of PSPNet and 4.2% of DeeplabV3+. The number of floating point computing is 7.989GFLOPs, only 6.7% of PSPNet’s floating point computing and 4.8% of DeeplabV3+. The mean intersection over union is 73.75%, slightly lower than PSPNet and DeeplabV3+. It achieves a better balance between computational efficiency and segmentation accuracy.