Yes, you will need to purchase a router in order to share the connection. Strictly speaking there are other ways of doing it, but if you can afford a router it will be much quicker and easier to set up.

To echo William Wilson, you will need to purchase a router in which you will be able to either plug both machines into or connect wirelessly depending on whether or not you have wireless network cards in both. From there, you will need a short network cable to go from the router (there is usually a separately marked port) into your modem's "Internet" port.

Assuming your new router supports DHCP, you shouldn't need to do anything other than connect everything and perhaps enable DHCP support via the networking TCP/IP properties on both machines.