An appeal to demolish a bungalow and build ten retirement flats has been refused an on the grounds the apartment block would be too big for the area.