This paper proposes an efficient, adaptive constraint handling approach that can be used within the class of evolutionary multi-objective optimization (EMO) algorithms. The proposed constraint handling approach is presented within the framework of one of the most successful algorithms i.e. multi-objective evolutionary algorithm based on decomposition (MOEA/D) [1]. The constraint handling mechanism adaptively decides on the violation threshold for comparison. The violation threshold is based on the type of constraints, size of the feasible space and the search outcome. Such a process intrinsically treats constraint violation and objective function values separately and adds a selection pressure, wherein infeasible solutions with violations less than the identified threshold are considered at par with feasible solutions. As illustrated, the constraint handling scheme extends the current capability of MOEA/D to deal with constraints. The performance of the algorithm is illustrated using 10 commonly studied benchmark problems and a real-world constraint optimization problem, and compared with the results obtained using yet another commonly used form i.e. Nondominated Sorting Genetic Algorithm (NSGA-II).