Finite element simulations of temper rolling of a steel sheet were performed using a model of elasto-viscoplasticity that describes the yield-point phenomena such as a high upper yield point, the rate-dependent Lüders strain at the yield plateau and the subsequent workhardening. From the simulations of uniaxial tension of temper-rolled sheets, the mechanism of elimination of the yield point by rolling has been clarified. For the purpose of the elimination of the yield point, the roughness of the roll plays an important role. During the temper rolling with rough surface rolls, a number of plastic zones are non-uniformly formed on the surfaces of the sheet, and from these numerous Lüders bands will start to propagate simultaneously. Consequently, macroscopic plastic deformation more likely to occur in the temper-rolled sheet under a comparatively low load without showing the yield point.