A method is proposed for designing three layer neural networks that gives relevant network structures which assure global minimization of learning errors for small training sets and small learning errors for big training sets both irrespective of the initial values. A condition on network structure is considered to achieve the above purpose, and a number of possible structures are provided together with their learning algorithms. Also, the generalization abilities of the network structures are discussed to guide the choice of structures in practice. All of the proposed structures for small training sets have zero errors after learning by a gradient-based algorithm and thus solve the local minima problem. The difference between them is in the level of locality and generalization abilities. For a big training set, first, the structure with zero learning errors for part of training data is obtained, then all of training data are used to train the network of the given structure, which improves the generalization abilities. Numerical examples are provided that support the present approach.