Sure, there is a difference. The numbers aren't there for fun, they do serve a purpose. The greatest difference for me is using a SATA 36GB WD Raptor (ATA150, 15K RPM) vs. a standard PATA 7.2K drive. Windows XP installs in around 15 minutes on the Raptor, around 40 minutes on the standard PATA.

Windows also boots up faster.

You will not notice a difference if you are only accessing Word documents or such from the slower drive. I store all my intensive programs on my Raptor (C:\) and use my larger PATA drives for storage. There is no noticable slow-down accessing a file from one drive over the other.