Dynamic Multi-objective Optimization (DMO) is a challenging research topic since the objective functions, constraints, and problem parameters may change over time. Several evolutionary algorithms have been proposed to deal with DMO problems. Nevertheless, they were restricted to unconstrained or domain constrained problems. In this work, we focus on the dynamicty of problem constraints along with time-varying objective functions. As this is a very recent research area, we have observed a lack of benchmarks that simultaneously take into account these characteristics. To fill this gap, we propose a set of test problems that extend a suite of static constrained multi-objective problems. Moreover, we propose a new version of the Dynamic Non dominated Sorting Genetic Algorithm II to deal with dynamic constraints by replacing the used constraint-handling mechanism by a more elaborated and self-adaptive penalty function. Empirical results show that our proposal is able to: (1) handle dynamic environments and track the changing Pareto front and (2) handle infeasible solutions in an effective and efficient manner which allows avoiding premature convergence. Moreover, the statistical analysis of the obtained results emphasize the advantages of our proposal over the original algorithm on both aspects of convergence and diversity on most test problems.