Council members expressed concern about the scale, setting and design of the proposed building.