Classification of the ultrasonic wave signals emitted from defects such as delamination and inclusions in layered media has been attempted using the neural network technique. In providing both reflected waveform from defects to the input and information of defect type, as well as location to the output, the relationships between the input and the output were learned beforehand by the neural network. In the system, the network learned in terms of detected waveform, however, could not infer correct information of defects for a waveform of a different phase. Therefore, appropriate parameters sampled from the detected waveform were added to the network. Locally connected networks restricting the connection between input layer and output one were proven in a highly precise estimation. It was revealed that the locally connected network is an effective technique to estimate the location of each defect, types of defect, and the amount of defects, within the range of the application of the model.