Recently, a general methodology for designing chaotic and hyperchaotic cryptosystems was developed using the control systems theory. Grassi proposed a nonlinear-observer-based decrypter for the state of the encrypter. If we can design the decrypter without the knowledge of the parameters of the encrypter, the chaos-based secure communication systems are not secure. In this paper, we design an observer-based chaotic communication system, which allows us to assign the relative degree and the zeros of its encrypter system. Moreover, under some conditions, we design an adaptive decrypter using the adaptive parameter adjustment law based on a Riccati equation in the case where the transfer function of the encrypter is of minimal-phase type.