I think rankings do serve some purpose but like everything else, we have to come to any conclusion after we look at the whole picture, including ranking.
To someone outside the US, rankings of US colleges/universities are even more helpful because Dartmouth, Rutgers, UPenn and PennState are all equally less known to them until they do their research.
However, like Alain says, what gets you the job is how you sell yourself at the interview. But given the programs are more or similar to your needs, if you get into a university which has lower acceptance rate, it tells the hiring manager something about you. [Ranking and acceptance rate may not be correlated]
Maybe hiring managers/recruiters can confirm this: If you graduate from a 'better known' university or a 'better known' program, it increases your chance of securing an opporunity to sell yourself to a manager.