Perlite will never be cheap. If you can't find it locally, you'll have to get it delivered, which will be expensive. The best option I found was greenhouse megastore. I only use the coarse kind (meaning larger pieces), which is a little harder to come by.
If you're looking for really cheap options for the soil, you can do what Jon or others said and probably get a load of decent stuff for dirt cheap (pun intended). But it will be heavy, and the bottom might get soggy, particularly in a large pot. I like lighter, better draining stuff, but others have reported success with these "heavy" mixes.
The largest component of my container mix (~80%) is pine bark fines, aka partially composted pine bark, or pine bark mulch. It's light (compared to dirt) and drains well. I can get a 2 or 3 cubic foot bag for about $5. Then I add some agricultural lime (to lower acidity, if necessary), some fertilizer, and some perlite. You should be able to find posts describing this mix if you do a search. One cubic foot is about 7.5 gallons, so all in the mix costs me maybe $4 per 10 gallon pot.