We propose an effective method of dynamic reorganization using reinforcement learning for the team formation in multi-agent systems (MAS). A task in MAS usually consists of a number of subtasks that require their own resources, and it has to be processed in the appropriate team whose agents have the sufficient resources. The resources required for tasks are often unknown \ extit{a priori} and it is also unknown whether their organization is appropriate to form teams for the given tasks or not. Therefore, their organization should be adopted according to the environment where agents are deployed. In this paper, we investigated how the structures of network and the number of tasks affect team formations of the agents. We will show that the utility and the success of the team formation is deeply affected by depth of the tree structure and number of tasks.