Crop growth was enhanced in fields previously cultivated with host plants colonized by arbuscular mycorrhizal fungi (AMF), compared with fields previously cultivated with non-mycorrhizal plants. To clarify the effect of previous cropping on the community structure of AMF in soybean roots, soybean were grown in fields which were cultivated after mycorrhizal plants, non-mycorrhizal plants, or left in uncropped condition over three years (in 2004, 2006, and 2007) in two different soils (Thaptoupland Wet Andosol and Low-humic Andosol). The partial region in the 18S rDNA of AMF from soybean roots was amplified by a nested PCR method using primers specific for AMF and sequenced. The sequence homology search and phylogenetic analysis revealed that the AMF community in soybean roots was unaffected by the preceding crop. Further, it was shown that the AMF phylotype “Glo-B1”, which included Glomus sp. ZJ (AB076344), was the most frequently detected, irrespective of the preceding cropping system. However, in 2007, the community structure of AMF in the soybean roots from the Low-humic Andosol field, which had been used as grassland for several years, was relatively different from that of the Thapto-upland Wet Andosol fields. It was implied that the AMF community could be affected by environmental condition or long-term vegetation.