Someone brought this up at a recent WS-I meeting and I thought it was funny enough to riff on; "Things with 'science' in their names usually aren't". Examples were provided such as "Political Science", "Social Science", and "Scientology" (the last one is a stretch). The shared joke being that we all felt our profession, despite outwards appearances, to be much more akin to political science than physics.
Certainly there are sub-fields of computer ccience that are scientifically rigorous, but I would guess that the majority of "programmers" rarely measure anything more than simple performance metrics, rarely use any math more complicated than basic combanitorics, etc. Obviously you need to be able to think logically and express your ideas in a non-ambiguous language in order to program, but that doesn't make us scientists any more than reheating frozen waffles makes someone a chef. I've always thought that Computer Science (the programming part - not the designing chips part) would be more properly thought of as a "Applied Philosophy" than as a sub-branch of mathematics, science, or engineering.