In this paper, a new hybrid of genetic algorithm (GA) and simulated annealing (SA), referred to as GSA, is presented. In this algorithm, SA is incorporated into GA to escape from the local optima. Then, the idea of hierarchical parallel GA is borrowed to parallelize GSA for the optimization of multimodal functions. In addition, multi-niche crowding is used to maintain the diversity in the population of parallel GSA. The performance of the proposed algorithms is evaluated against a standard set of multimodal benchmark functions. Multi-niche crowding PGSA and normal PGSA show some remarkable improvement in comparison with the conventional parallel GA and the breeder genetic algorithm.