The most complicated thing about football is the pass interference rule. I have no idea how that works unless it is really obvious.
Whatever the rule book says, it's mostly a matter of the ref's common sense--which means you see some variation in how the rule is applied. The first thing to know is that interference can only happen:
(1) when the pass is forward (i.e., not on a lateral)
(2) while the ball is in the air (not before; hitting a receiver before the pass is legal inside 5 yards; outside 5 yards, it's illegal contact, not interference)
(3) before the pass is touched (so if it's tipped, anything goes)
(4) when the ball is catchable by one of the parties involved (i.e., contact on another part of the field can't be called pass interference)
The basic ideas are these:
Was the player going for the ball? If not, then it's interference.
Was one player in a clearly worse position to make a play on the ball than the other? If so, then it's almost always interference.
There is a bias in favor of the offensive player, since it's harder to catch a pass than it is to tip one away.
Pass interference, like all penalty calls (and non-calls), is not reviewable by instant replay.
So there are the basics. NFL corners have developed some effective tricks--like turning the receiver's hips at the last instant--that are very difficult to see at game speed, especially if you're watching the ball. This can make some interference calls non-obvious until you see a replay shot from the correct angle.
At the same time, receivers have become far more physical than was once the case. Some wideouts and many tight ends will actually initiate contact in order to gain position. In those cases, offensive interference is rarely called; however, the defender is usually allowed to respond in kind without being flagged.