The Harmony Search (HS) method is an emerging meta-heuristic optimization algorithm. In this chapter, we propose two modified HS methods to handle the multi-modal and constrained optimization problems. The first modified HS method employs a novel HS memory management approach to handle the multi-modal problems. The second modified HS method utilizes the Pareto-dominance technique, and it targets at the constrained problems. Several simulation examples are used to demonstrate and verify the effectiveness of our new HS methods.