Surrogate-assisted evolutionary multiobjective optimization algorithms are often used to solve computationally expensive problems. But their efficacy on handling constrained optimization problems having more than three objectives has not been widely studied. Particularly the issue of how feasible and infeasible solutions are handled in generating a data set for training a surrogate has not received much attention. In this paper, we use a recently proposed Kriging-assisted evolutionary algorithm for many-objective optimization and investigate the effect of infeasible solutions on the performance of the surrogates. We assume that constraint functions are computationally inexpensive and consider different ways of handling feasible and infeasible solutions for training the surrogate and examine them on different benchmark problems. Results on the comparison with a reference vector guided evolutionary algorithm show that it is vital for the success of the surrogate to properly deal with infeasible solutions.