In this paper, we consider a stabilization problem of a mobile robot in an unknown plane at an unknown destination. For the problem, we propose a reinforcement learning based feedback controller design method on the unknown plane. We can design a semi-global CLF with an optimal action-value function obtained by reinforcement learning and local CLFs via multilayer minimum projection method. The feedback controller is designed by the semi-global CLF. The effectiveness of the proposed method is shown by computer simulation.