MOEA/D is an evolutionary multi-objective optimization algorithm, which relies on decomposition methods such as, weighted-sum, Tchebycheff and Penalty-based Boundary Intersection (PBI) to convert a multi-objective problem into a set of single-objective problems. It is known that PBI can generate a more uniform set of solutions than the other decomposition methods. The drawback of PBI is that it has a penalty parameter (θ) that has to be specified by the user. This penalty parameter can affect the convergence rate of MOEA/D as well as the uniformity of solutions. Unfortunately, there are very limited studies on sensitivity analysis of MOEA/D on the penalty parameter of PBI. This paper is dedicated to a comprehensive analysis of PBI's penalty parameter, and its effect on a user-preference algorithm (R-MEAD2) and a non-user-preference algorithm (MOEA/D). Unlike the previous studies that only rely on Hypervolume as their performance measure, we study the effect of θ on convergence, uniformity, and the combination of convergence and uniformity independently. The experimental results suggest that user-preference algorithms consistently perform better with a relatively larger θ value as compared to their non-user-preference counterparts. The results also suggest that on some problems, such as multi-modal functions, convergence is the dominant factor on the overall performance, where a smaller θ is preferable. Conversely, on some other problems, a larger θ is suggested where uniformity is the dominant factor. Finally, we briefly investigate the relationship between θ and the number of objectives.