Lots of decent points so far.
My experiences, with almost 60,000 rounds reloaded since I started a few years ago:
I started on a Dillon 550 and added a 650 later. You can learn on a progressive, but just like with single stages don't rush things and have NO distractions in the room with you.
Now I load 9mm, 45 and 223 in quantity on the 650 (a casefeeder = spoiled reloading brat), and 44mag, 45-70 govt on the 550.
I also have a single stage press, a cheap but quite solid Lee classic(?), a sturdy cast iron utility tool. I've used it for a lot of things including pulling bullets from mistakes and reclaimed rounds, fixing crimps on loose 223 due to processed brass that had a too wide case mouth
I cast for all my reloading calibers except 223 quite successfully, including 9mm. 44 mag hot loads are the only ones I need gas checks on
smurf is right on the misconceptions about hard bullet alloy = less leading, quite the opposite in most cased and this is where people give up. I have gravitated to an alloy mix that is softer than standard wheel weights and thanks to good fits with the bores of my guns and careful load tweeking and documentation until I found to perfect mix.
Cast bullets are easier on your guns barrels than jacketed and are more accurate (once you find the formulas for each gun)
remember components don't go bad. shelf life is effectively forever under reasonable conditions.
If treat it like a lifetime scavenger hunt and let your friends and peers know that you reload and/or cast, you'd be surprised what comes your way. I have had a lot of free lead sources materialize this way and have only tried tire store WW begging twice (years ago my-ex tired store wouldn't even sell me one bucket, note "ex")
wheel weights are everywhere in the road and gutter and crosswalks. keep your eye out when you are on foot and just start picking them up. Between mrs cohutt and I, we collect about 30 or 40 lbs per year this way out of habit. (OK maybe a little ocd but you have to be committed lol)
buy more components steadily even when you have a good reserve, it probably means supply is good for a change and you will never regret it. components don't lose value unless purchased in haste during a panic (ie possibly today). remember they don't go bad.
Join gun forums where reloading is popular and check the buy/sell/trade boards regularly. You can almost always find fair prices in these and sometimes great deals. i've made trading deals and friends on ar15.om, glocktalk and glockpost as well as castboolits.
if you find a deal on components common calibers, jump on it especially if an almost free give away (ie yard sales and general "my ex left me this buy if fast please and get it out of my sight" situations). Even if you don't load those calibers there are a lot that do, see previous point about trading/selling in forums. I manage to accumulate a lot of 40 brass snarfing up stuff at the range; i don't have or want a 40 and have traded the brass for components i do use on several occasions.
Which reminds me, I have about 4 or 5 gallons of cleaned 40 brass in good shape if anyone is looking for it, lets talk
pick up all brass at all ranges where you shoot, let your non reloading friends know you reload and to please save all their brass. you can sort through calibers and condition of brass later, just pick it up when you can find it. see previous points
above all remember free beats cheap every day of the week.