Bilevel optimization is a type of mathematical program in which one optimization problem (called the lower level problem) is nested within another (called the upper level problem). In recent years, there has been considerable interest in the development of algorithms that can handle multiple objective functions at both levels. The challenge lies in the implication that every upper level decision leads to a set of Pareto optimal solutions at the lower level. As a result, a single point in the upper level decision space maps to a set of points in the upper level objective space. Since standard multiobjective evolutionary algorithms (MOEAs) are not designed for such point-to-set mappings, the state-of-the-art solution methods often dictate several enhancements to existing MOEAs. In this paper, we propose an adaptive scalarization based approach to solving bilevel programs with multiple objectives at both levels, such that any off-the-shelf MOEA can be used with minimum modification. Subsequently, we put forward a surrogate-assistance technique that can significantly lower the computational cost commonly associated with such problems. Finally, proof-of-concept numerical experiments are carried out in order to demonstrate the potential of our proposed methods.