Check out this Youtube video. Warning: it's 15 minutes long.
(Boris, I'd like to discuss swimming, cuz he doesn't mention several new rules.)
It's interesting - I talk about this with people from time to time. It's been a while since I've seen this video or read anything about it (and I don't feel like watching it again) so I apologize if I miss major points.
I'd say, yes, swimmers are generally getting faster. The top of the top at all levels is stronger across the board. Expectations have changed and that shifts the psychological game - if the expectation is that, for example, a sub 20second 50yard freestyle is very attainable, you'll have more people attain it.
I coach a high school girls team. Many of the girls on my team now would have been strong performers on my high school (boys) team in the 80s. BUT, overall a good/solid swimmer from the 70s or 80s would still be a good/solid swimmer now.
#1) There are A LOT more teams, more swimmers, and more and better facilities for said teams and swimmers. If you have more kids getting into the sport, you're starting with a larger 'pool' of athletes to work and after you sift out the weak, you're going find some real diamonds. The mutants are mutants - no arguing that.
#2) In a sport where a second can mean a lot, suit and goggle technology matters. They got rid of the full body shark skin suits for a reason, and although they dialed it back a lot, the coverage is still significantly more than it was 20 or so years ago (pre-full body wet suit era).
#3) Streamline & underwater dolphin-kick. There were huge time drops after everyone started doing underwater dolphin kicking. Short butterfly, backstroke, and freestyle events became an underwater dolphin contests until they limited it to 15 yd/m. I don't think any world class swimmers do a flutter kick off the wall anymore - like, no one.
#4) Backstroke turns. It used to be that you had to touch the wall w. your hand prior to turning. You don't have to anymore. I don't know when that changed, but it's huge time advantage equivalent to swimming a meter less each length.
#5) Strength Training. Swim coaches are a pretty conservative bunch, but they've started to come around in this area.
#6) Pool technology. Pools are bigger and faster. A bigger/better pool disperses waves better. Less waves = less interference for swimmers and faster swimming. There are probably other things in this area that I just haven't bothered to keep up with...