Scientists and engineers agree that solving complex problems requires integrating traditional physics-based modeling techniques with state-of-the-art deep learning (DL) methods. This paper aims to integrate physics knowledge into a convolutional neural network (CNN) to boost learning within a feasible solution space in a specific domain. Our proposed method uses deep neural networks in the form of (CNNs) augmented with custom loss functions which uses physics rules to bypass the need for Finite Element Analysis and predict high-resolution stress distributions on damaged steel plates with variable loading and boundary conditions. We embedded physics constraints into the loss function to enforce the model training, precisely capturing stress concentrations around the tips of various structural damage configurations. The CNN was designed and trained to use the geometry, boundary conditions, and load as input and predict the stress contours. The proposed framework’s performance is compared to Finite-Element simulations using partial differential equation (PDE) solver. The trained DL model can predict the stress distributions of damaged steel plates with a mean absolute error of 0.22% percent and an absolute peak error of 1.5% for the Von Mises stress distribution.