Many workers will be able to accumulate $1 million over the course of their career if they save consistently beginning at a young age, invest prudently, and avoid withdrawing money early. It helps, of course, to sidestep fees, taxes, and penalties along the way whenever possible. Here’s what it takes to become a millionaire by retirement:

Start saving by age 25. It’s difficult to start saving for retirement when your entry-level salary barely covers your student-loan payments. But beginning to build a nest egg during your first job is the most painless way to become a millionaire by retirement. “The more time you have, the less you have to save each year since you have longer to accumulate,” says Kim Morton, a certified financial planner for Sensible Money in Scottsdale, Ariz. If you start saving for retirement at age 25, you only have to save about $4,830 annually to reach $1 million by age 65, assuming an annual return of 7 percent after fees. If you wait until age 40 to start saving, you’ll need to tuck away much more: $15,240 per year, assuming the same retirement age and annual return. Alternatively, you could start out saving slightly less and boost your savings rate as you receive raises and bonuses. “A 20-year-old may only be able to save $100 per month, but the exercise of saving $100 per month when you are 20 will make it easier for you to go to $200 a month when you are 25 or 30,” says William Valentine, a financial planner and founder of Valentine Ventures in Bend, Ore. “The act of saving is like developing a muscle, and if you start early with small amounts, you will build the saving muscle.”

Select low-cost investments. Fees incurred through your 401(k) plan will cut into your investment returns. A 401(k) contribution of about $7,795 per year over 35 years will get you to $1 million, assuming a 7 percent annual return and fees and expenses of 0.5 percent. However, if your fees are 1 percent higher (1.5 percent total), you will need to save $9,690 per year to become a millionaire over 35 years, or about $1,895 extra per year to make up for the higher fees. “If I have the choice of two mutual funds that have the same characteristics, but one has [an expense ratio] of 1.5 percent, but the other [charges] 0.5 percent, by using the fund with an expense ratio of 0.5 percent, it is like saying, ‘Here is your extra 1 percent of returns each year’,” says Morton. “Simply put, the less money you put in somebody else’s pocket when it comes to fees, taxes, and penalties, the more money you will have in yours.”

Get a match. A 401(k) match will help you become a millionaire significantly faster than you could on your own. If you get an annual $1,500 401(k) match from your employer, you can save $1 million by contributing just $5,475 annually to a 401(k) plan for 35 years, assuming a 7 percent annual return after fees, versus $6,975 annually without the match. But pay attention to your employer’s vesting schedule for the retirement plan. Until you are fully invested in the plan, you may not get to keep employer contributions to your retirement account. In some cases, you might need to remain with an employer for five or six years until you can keep your 401(k) match.

Mind retirement benefit gaps. Saving for retirement becomes much more difficult when people change jobs, get laid off, and take time out of the workforce. And when you get your next job, there may be a waiting period before you can join the 401(k) plan or get a 401(k) match. To keep your retirement plan on track, you might need to save in an IRA or a taxable investment account until you become eligible for a new retirement plan at work. “Try to save equally outside of work what you would do at work,” says Valentine. If that’s not possible, you might need to save extra in advance of or after a retirement savings break.

No early withdrawals. One of the biggest obstacles to reaching $1 million is taking money out of your 401(k) before retirement. Whenever you withdraw money from a 401(k) account, you will have to pay income tax on the amount withdrawn. Those who withdraw money before age 59½ are also generally charged a 10 percent early withdrawal penalty. For example, if you withdraw $10,000 from your 401(k) plan at age 40 and are in the 25-percent tax bracket, you will forfeit $3,500 of that amount in taxes and penalties. Your money will grow faster if you avoid the early withdrawal penalty and defer tax on your savings until retirement. Avoid cashing out your 401(k) when you change jobs by rolling the money into an IRA or your new employer’s 401(k) plan, or even leaving it in your old 401(k). An emergency fund outside of your retirement account will give you a financial cushion so that you don’t have to dip into your retirement savings for unexpected expenses.

Balance safety with growth. Unless you have significant investment expertise, your 401(k) portfolio probably won’t outperform the stock market every single year. Most people should aim to capture the average growth of the stock market. “Return does not mean trying to get some crazy 20 percent return each year because somebody told you that it’s possible,” says Morton. “Return means that there is going to be some type of average market return and that the longer you have your money invested, the closer you are going to get to that average return.” Once you start to accumulate a significant account balance, you need to protect it with a reasonable investment strategy that includes a mix of stocks, bonds, and cash that is appropriate for your risk tolerance.

Boost savings once your kids are independent. Once your children are finished with college and support themselves, you will have a newfound ability to tuck money away for retirement. “Usually people don’t have a lot of money when the children are in school. You usually find that the period of time in your 50s and your mid-60s is when you are really putting away a lot of money,” says Harold Anderson, a certified financial planner and president of Parkshore Wealth Management in Roseville, Calif. “I think if each member of a couple is putting $22,500 in a 401(k) and investing it in a reasonable manner, they could probably have a pretty good shot at getting close to a million.”