In mobile communication environment, the wireless channel is time varying due to the mobility of the wireless terminal and multi-path propagation. To provide wide area network coverage at true broadband data rates, both WiMAX and future versions of 3GPP must adopt Multiple Input Multiple Output (MIMO) and Adaptive Antenna Systems (AAS) technologies. So an algorithm for reconfigurable communication system based on both MIMO and AAS is proposed. In the proposed algorithm, either MIMO or AAS will used at a time according to the environment and has been experimented with direction of arrival estimation, beamforming techniques and also the performance of MIMO system analysis carried out in high scattering environment. Finally this paper investigates antenna grouping algorithms, which are hybrids of beamforming and spatial multiplexing. The performances of antenna grouping and MIMO systems arrays are evaluated using capacity as the metric.