Download a PDF version of this article

You’ve probably thought about how much you’ll need to retire comfortably. But if you haven’t done the calculations, you’ll likely be shocked at the actual amount. Most physicians need $5 million-$6 million in investable, liquid (cash) assets to retire comfortably based on a 3%-5% after-tax annual return.

Most doctors expect to do a lot better than 5% after tax on their portfolios. But most doctors to whom I’ve spoken at investment seminars tend to overestimate what they will earn in the markets.

Realistically during the past 100 years, bonds have returned 5%-6% and stocks about 9% annually before taxes. So based on past performance, these averages are unlikely to change much in the next 100 years. Despite this, many investors base their retirement plans on annual investment returns of at least 15%, which is historically an unrealistic expectation.

There is also the inflation factor. Even at a modest 3% annual inflation, a physician who currently lives on a $10,000 monthly income will need $16,000 per month 12 years from now just to maintain the same lifestyle. It would take at least $4 million in invested funds at 4% annual tax-free return to generate $16,000 per month, and bear in mind that amount does not provide for any significant increases in personal health expenses, which tend to increase with age.

Plan for occasional loss

Aside from planning for increases in your health expenses, also anticipate occasional losses with your investment portfolio. Consider that a $1 million portfolio growing at 6% annually takes 12 years to become $2 million – and that projection assumes no losing years. Realistically, most portfolios suffer occasional years of loss, and making up for a losing year can take far longer than most investors expect.

I find that most physician investors have not considered the effect an occasional annual loss can have on their retirement portfolios. Take a physician investor with a high tolerance for risk and an aggressive portfolio that gains 50% in a year when the market gains only 8%. The following year, the doctor’s aggressive tactics backfire and the portfolio loses 50%.

At least the doctor didn’t lose any money, right?

Wrong. In this example, a portfolio starting with an investable asset base of $100,000 ends year one at $150,000. But in year two the 50% loss leaves the portfolio with just $75,000 – a 34% loss. Suppose, at this point, the investor loses a taste for whichever aggressive investment strategy resulted in the portfolio loss. Now that physician investor switches to a more conservative strategy that generates a steady 9% annual return rate.

At this point, getting back to the investor’s original $100,000 starting point will take four more years. Over six years, factoring in inflation, the portfolio will wind up with less than net zero performance. This physician investor is now farther away from the retirement goal than six years ago and the loss of compounded interest can’t be reclaimed. It doesn’t take many losing years to cripple a retirement planning effort.

Gauge risk vs. return

The amount of risk taken to achieve a given return is as, if not more, important than the returns achieved. For example, during the same period mentioned above, a risk-averse physician investor with a $100,000 portfolio and a steady annual return of 5% would reach more than $160,000 during that six-year span.

Always safeguard your investable asset base so you don’t end up with less money than when you started – there is no opportunity for future returns without investable assets. Obtaining high returns should be secondary, especially when investing for your retirement.

Note that when it comes to determining your investment risk, put as little of your capital at risk as possible. Most years, it’s better to garner a 15% return with only 30% of your capital at risk than a 20% return while risking 100% of your capital. If a natural disaster, act of terrorism, currency devaluation, inflation or interest rate spike, or other event creates a market tremor, portfolios with the least amount of capital at risk are the best insulated against loss.

Create an advisory team

To help you avoid annual portfolio losses, build what Warren Buffet, chair of the holding company Berkshire Hathaway, Inc., and dubbed one of the greatest investors of all time, calls a “circle of competence.” This circle is a team of competent advisors to help you reach your retirement objectives.

A key member of this team is an investment advisor who constructs various retirement scenarios, determines necessary annual contributions, and manages assets within your personal risk tolerance and investment model preferences. The circle may also include a certified public accountant, a tax advisor, or a tax attorney.

When selecting an investment advisor, many investors look for someone who already manages large amounts of money – this person seems like a safe choice because thousands of other investors already rely on him or her. But traditionally, the more money a manager takes in, the harder it is to deliver performance above broad market returns. Although investors tend to focus on a manager’s returns in positive years, I recommend looking at a manager’s history of avoiding losing years. After all, as you have seen from the example above, investment returns that soar 50% one year and sink 50% the next can crush your portfolio.

After you choose your investment advisor and the other members of your team, make your portfolio goals clear. Also assemble a complete asset inventory (e.g., money, securities, equipment, and buildings) so your team can use their financial knowledge and resources accordingly. For example, you may want to look for ways to improve undervalued assets to add more income into your retirement plan because all gains are tax-deferred.

Say your practice owns a professional building. Perhaps you can purchase it and use resources generated by that into your retirement plan. This strategy shelters the income generated as well as the proceeds if you sell the building. Also, debt on assets such as property can be financed by a pension plan, generating tax-free interest that is nonetheless deductible.

The longer you postpone an analysis of your retirement needs, the more difficult it becomes to save enough money for your projected retirement date. Remember, you will probably need more money than you thought to retire and still maintain your current lifestyle, and your investable assets may not grow at the rate of annual interest you thought. But with an investment strategy that embraces a long-term approach, avoids annual losses, and complements your tolerance for risk, you can reach retirement with sufficient assets to maintain your hard-earned lifestyle.