Mortgage Payoff with Retirement Savings

In retirement planning with our clients we often come across questions that require some personal planning and customized consideration. In other words, not everyone’s situation is the same. Here we analyze the various considerations one should think about when approaching retirement and looking to eliminate their mortgage. There are some understandable questions you might encounter as you plan for retirement. Leading up to retirement you will likely be stashing away money in an employer-sponsored retirement plan such as a 401(k) or 457 Deferred Comp plan while simultaneously making a hefty monthly mortgage payment? So could it be better, in the long run, at or close to retirement to use retirement savings to pay down the mortgage? That way, you'd substantially reduce your monthly expenses before you leave behind work and its regular paychecks.

KEY Conisderations in this BLOG

• Ideally, paying down a mortgage with funds from your 401(k) or 457 can reduce your monthly expenses as retirement approaches or begins.
• A pay-down can also allow you to stop paying interest on the mortgage, especially if it's fairly early in the term of your mortgage. We will analyze this in more detail.
• Significant disadvantages to the withdrawing a large lump sum include reduced assets in retirement and a higher tax bill in the year in which the funds are withdrawn from the 401(k) or 457.
• You'll also miss out on the tax-sheltered investment earnings you'd make if the funds remain in your retirement account. The compounding growth of the larger asset base.

There's no single answer as to whether it's prudent to discharge your mortgage prior to retirement. The merits depend on your financial circumstances and priorities. With proper planning we can help you make the most informed decision.  Here are some of the pros and (compelling) cons of the move, to help you decide whether it might make sense for you.

Pros

• Increased cash flow
• Elimination of interest to mortgage company or bank
• Estate-planning benefits
• Possibly mortgage interest deduction if you itemize. If you use the current standard deduction the mortgage interest might be a non-factor. Consult with your CPA.

Cons

• Reduced retirement assets and tough to build back
• A hefty tax bill
• Loss of mortgage-interest deductibility
• Decreased investment earnings on a larger asset

Pros to Discharging Your Mortgage

Here are the factors in favor of living mortgage-free in retirement, even if it means using up much or all of your 401(k) balance in order to do so.

Increased Cash Flow

Since a mortgage payment is typically a hefty monthly expense, eliminating it frees up cash for other uses. The specific benefits vary by the age of the mortgage holder.

For younger investors, eliminating the monthly mortgage payment by tapping 401(k) assets frees up cash that can be used to meet such other financial objectives as funding college expenses for children or purchasing a vacation property. With time on their side, younger workers also have optimal ability to replenish the drawdown of retirement savings in a 401(k) over the course of their working years.

For older individuals or couples, paying off the mortgage can trade savings for lower expenses as retirement approaches or begins. Those reduced expenses may mean that the 401(k) or 457 distribution used to pay off the mortgage needn't necessarily be replenished prior to leaving the workforce. Consequently, the benefit of the mortgage payoff persists, leaving the individual or couple with a smaller need to draw income from investment or retirement assets throughout retirement years. The excess cash from not having a mortgage payment may also prove beneficial for unexpected expenses that could arise during retirement, such as medical or long-term care costs not covered by insurance.

Elimination of Interest

Another advantage of withdrawing funds from a 401(k) or 457 to pay down a mortgage balance is a potential reduction in interest payments to a mortgage lender. Over the course of a conventional 30-year mortgage on a $200,000 home, assuming a 3.25% fixed interest rate, total interest payments equal slightly more than $170,000 in addition to the principal balance. Utilizing funds from a 401(k) to pay off a mortgage early results in less total interest paid to the lender over time.

However, this advantage is strongest if you're barely into your mortgage term. If you're instead deep into paying the mortgage off, you've likely already paid the bulk of the interest you owe. That's because paying off interest is front-loaded over the term of the loan.

Just because you are 10 years into a 20-year mortgage for $300,000 doesn't mean you now owe the bank $150,000. Mortgages don't work in a linear fashion like that. You have spent the last 10 years predominantly paying interest and still owe principal that is going be considerably more than half what the original loan was for. The 10 years that lie ahead of you comprise paying down more and more principal and less and less interest with each payment."

Put another way, having paid off almost all the interest on your mortgage earlier in its term, the continuing cost of borrowing the principal that remains—which may be most of that sum—is essentially negligible, a gift you should reject only with much thought. We feel you have already done much of the heavy lifting regarding interest payments, meaning if you suddenly give the bank the principal owed now, the effective interest rate that you will have paid over the [now-shortened] life of that mortgage would probably shock you if you calculated it.

Look at this example:

Original Loan Amount $250,000, Original Term-15 years, Rate-3.2%, Years remaining- 8 years

After 7 years in and 8 years remaining the current outstanding balance would be  $148,075.You have paid a total of $147,000 after 84 monthly payments. This includes $45,075 in interest. If you make regular monthly payments you will pay an  additional $19,925 in interest over the next 96 months.

As you see the interest one will save would be $19,925. Also, if the individual desired to take out $148,075 out of our retirement account we need to identify the gross withdrawal from a retirement plan needed to net $148,075. It could be close to $190,000 at a federal tax withholding of 20%. How long will it take to make that up that savings. Lastly, there is a need to analyze the growth of the asset if no withdrawal or minimal amount to supplement monthly payment instead of a lump sum.

Estate Planning

Additionally, owning a home outright can be beneficial when structuring an estate plan, making it easier for spouses and heirs to receive property at full value, especially when other assets are spent down prior to death. The asset-protection benefits of paying down a mortgage balance may far outweigh the reduction in retirement assets from a 401(k) withdrawal.

Cons to Discharging Your Mortgage

Against those advantages of paying off your mortgage are a number of downsides—many of them related to caveats or weaknesses to the pluses we noted above.

Reduced Retirement Assets

The greatest caveat to using 401(k) or 457 Deferred Comp funds to eliminate a mortgage balance is the stark reduction in total resources available to you during retirement. True, your budgetary needs will be more modest without your monthly mortgage payment, but they will still be significant. Saving toward retirement is an overwhelming task for most, even when a 401(k) is available. Savers must find methods to outpace inflation while balancing the risk of retirement plan investments. Contribution limits are in place that cap the total amount that can be saved in any given year, further increasing the challenge. The IRS limits contributions to $19,500 for 2021 for 401(k) plans, while the catch-up contribution limit is $6,500 for people aged 50 or over. And with the passing of the Setting Every Community Up for Retirement Enhancement (SECURE) Act in December 2019, you can now contribute past the age of 70½. That's because the act allows plan participants to begin taking required minimum distributions (RMDs) at age 72.

Due to these restrictions, a reduction in a 401(k) balance may be nearly impossible to make up before retirement begins. That's especially true for workers who are middle-aged or older, and therefore have a shorter savings runway in which to replenish their retirement accounts. The cash flow increase resulting from no longer having a mortgage payment may be quickly depleted due to increased savings to make up a retirement plan deficit.

A Hefty Tax Bill

If you're already retired, there is a different kind of negative tax implication. Overlooking the tax consequences of paying off a mortgage from a 401(k) or 457 could be one of the biggest mistakes we see retirees make. Any money taken out of the 401(k) is counted as ordinary income. Taking out a large sum to pay off something like a mortgage, especially during a year when the retiree still has earned income, can easily step a person into a higher marginal tax bracket and an unexpected tax bill. Please consult with your CPA.

For example, for a $100,000 distribution. Withdrawing that amount could easily move a middle-income earner from what could have been a 15% bracket to a 25% bracket. The distribution would then cost the person $10,000 in additional taxes—$13,000 if they made slightly more and it stepped them into the 28% bracket. We have seen people step themselves all the way up to the 39.6% bracket

The tax scenario is no better if you borrow from your 401(k) in order to discharge the mortgage, rather than withdraw the funds outright from the account. Withdrawing funds from a 401(k) can be done through a 401(k) loan while an employee is still employed with the company offering the plan or as a distribution from the account.

Taking a loan against a 401(k) not only requires repayment through paycheck deferrals but may also result in costly tax implications for the account owner. This happens if an employee leaves their employer before repaying the loan against their 401(k). In this situation, the remaining balance is considered a taxable distribution unless it is paid off by the due date of their federal income tax, including extensions. Similarly, employees taking a distribution from a current or former 401(k) plan must report it as a taxable event if the funds were contributed on a pretax basis. For individuals making a withdrawal prior to age 59½, a penalty tax of 10% is assessed on the amount received in addition to the income tax due.⁸

The Loss of Mortgage-Interest Deductibility

In addition to tax implications for loans and distributions, homeowners may lose valuable tax savings when paying off a mortgage balance early. Mortgage interest paid throughout the year is tax-deductible to the homeowner, and the loss of this benefit may result in a substantial difference in tax savings once a mortgage balance is paid in full.

It's true, as we noted earlier, that if you're well along in your mortgage term, much of your monthly payment pays down principal rather than interest, and so is limited in its deductibility. Nonetheless, homeowners—especially those with little time left in their mortgage term—should carefully weigh the tax implications of paying off a mortgage balance with 401(k) funds before taking a loan or distribution to do so.

Decreased Investment Earnings

Homeowners should also consider the opportunity cost related to paying off a mortgage balance with 401(k) assets. Retirement savings plans offer a wide array of investment options meant to provide a way in which returns are generated at a greater rate than inflation and other cash equivalent securities. A 401(k) also provides for compound interest on those returns because taxes on gains are deferred until the money is withdrawn during retirement years.

Typically, mortgage interest rates are far lower than what the broad market generates as a return, making a withdrawal to pay down mortgage debt less advantageous over the long term. When funds are taken out of a 401(k) to pay off a mortgage balance, the investment opportunity on these assets is lost until they are replenished, if they are replenished at all.

The Bottom Line

Keep in mind that you enjoy the likely appreciation in the value of your home regardless of whether you've discharged its mortgage. Financially, you might be better off overall to leave the funds in your 401(k) or 457 and enjoy both their possible appreciation and that of your home. Leveraging both asset values over time.

Sources: Investopedia