A smart server monitoring system is proposed to detect the working status of the server in real time, which finishes the recognition of the status of the server and indicator light through image recognition based on convolutional neural network. In order to solve the problem of class imbalance in target recognition, a novel loss function is designed to adjust class imbalances automatically. Its test accuracy of VOC2007 is 78.3%, which is better than the advanced methods such as YOLOv2. In addition, an image synthesis method based on multi-scale edge information fusion algorithm is proposed to reduce the cost of data collection. Its pixel precision of Cityscapes is 71%, which is much better than pix2pix.