This is an extremely large topic. A car has so many parts and everyone of them can be modified to increase speed. Many people, however, look for a cheap and inexpensive way to increase horsepower. The two most common ways people increase power are by using a cold air intake and an exhaust.
After market intakes are less constricting, with larger filters and more manderally bent tubing. Specially designed intakes, known as cold air intakes, not only increase flow but they pull cold air from under the bumper or car. They move the intake charge further away from the hot engine. This simple concept is that colder air is denser and therefore more air leads to more power.
Now to exhausts. Cat-back exhaust systems usually have smoother bends then standard and they will have larger diameters. They increase flow and allow the engine to breath. The mufflers at the back are smaller and many have straight through designs. One side effect is that the faster the air moves the larger the exhaust note becomes.
Adding downpipes, a.k.a. headers, along with a performance catalitic converter can add power as well. Stock cars have an exhaust manifold much like an intake manifold. The exhaust manifold is simply a small air chamber, with as many pipes going into it as however many cylinders u have. These pipes are perpendicular to the air chamber. The chamber the empties into a small pipe out to the muffler. This if very constrictive. The small piping and 90 degree angles create alot of back pressure. Headers differ in that each cylinder has its own manderally bent pipe, usually 1 foot or longer, that joins in a collector that opens up into it the catalitic converter or straight to the muffler.
Back to the basics:
There is one basic way to increase horsepower. Make a larger more powerful explosion in the combustion chamber. There are many ways to do this, however, they all have the same concept and/or purpose. Most parts regulate the air coming and going through the cylinders. For example, the intake and exhaust I just talked about. When increasing the amount of air that goes into each cylinder, you can increase the amount of fuel. This results in a power gain.
In a stock vehicle the amount of additional horsepower you can create is limited due to the strength of the block and cylinder heads as well as other key engine components. It is costly and time consuming to replace these engine parts.
Without doing internal engine work here are a few examples of what you could do to increase horsepower:
Change cam sprockets and/or cam shaft for an after market one. For more information about valve train click here.
Change your underdrive pulleys with lighter ones. Underdrive pulleys would be those of your water pump, alternator, and all other accessory pulleys including the crank pulley that runs these. Gains are a 5-10 hp.
A cold air intake and/or exhaust as described above.
Revamp your ECU. Gains depend on car and what chips you use. I have seen as much as 50hp or more gained from one of these.
Use a stronger ignition system. An example would be to look at MSD ignitions. Replacing stock spark plugs and wires would also be beneficial. A longer, stronger spark means you can add more fuel and have a better burn.
Forced Induction is another common and easy way to massively increase hp and torque of your car. This is relatively cheap when you compare it to other parts in a horsepower per dollar ratio though kits run about 3000 dollars and up. Forced induction has been known to increase power by 30 to 50 percent with only a minor kit. More is explained about forced induction at the following pages: Turbocharger Supercharger
If you are, however, willing to spend a lot of money (and time if you plan to do it yourself) internal engine modifications can be made to increase horsepower greatly. I will list a few examples:
Do a port and polish on your cylinder head to decrease turbulence in the air flow. You can buy new ported and polish heads that are stronger and lighter then your stock heads.
Increase valve size, get strong valve springs, or invest in some aluminum rocker arms.
Have an engine build up. Start from a stronger bottom end. Get better alloy pistons to with stand heat and pressure. Increase displacement using a different crank and conrods and etc. This is basically what you would have to do inorder to go to the extremes. Which include massive turbos or a 500hp shot of Nitrous. Any of these things that I listed can be done by itself or in combination. A stock engine usually can only with stand a 75hp gain. With these modifications you can do a lot more. However it cost a lot more.