Monday, July 1, 2019

So We're Going to Pay Off the Mortgage

So We're Going to Pay Off the Mortgage
Talking things over with a few friends last week helped us make a decision: we're going to pay off the mortgage. All at once, by using the remaining cash we have from selling our two rental properties, but mostly by selling investments we have in our taxable accounts.

I already know from our mortgage swoop posts that most people probably won't like this idea. I'll try my best to explain our rationale, which I really do think is sound. But I'm not trying to convince other people that this is what they should do. Instead, I want to explain where we're coming from, our thought process, and just want to be as transparent and honest as I'm comfortable with on the blog.

Transparency hasn't always been my approach. The last time we paid off a mortgage, with our first home back in 2013, I wrote about it and the post got a little publicity. A reporter from a major outlet interviewed me about it and then, the next day, I emailed her and asked if she would kill the story. 

Mrs. Done by Forty and I were convinced that someone would piece together the details, and our anonymity would be blown. What if our real names got out there? Would one of the right wing jagoffs I argue with online end up finding our home address? A lot of these dudes have guns. Some people happily threaten violence over Twitter. What if I pissed off the wrong dude?

Thankfully, the reporter was willing to stop the story. That scare nonetheless started a trend of being less and less open on the blog. I took down the post outlining our mortgage payoff. Weeks later, I'd deleted all our net worth and monthly 'budget porn' posts. I stopped revealing as much about our financial lives, and any thought of meeting other bloggers at conferences like FinCon or at meetups was out of the question.

Recently we've started realizing that our anxiety might not be entirely justified. No one really cares about our situation, as far as I can tell. And with other bloggers inspiring us to be more transparent, to be more honest about our situations, we're more comfortable sharing some of the details.

So yeah, about that mortgage payoff.

Here's the basics of our current situation:
  • $262,000 owed on the mortgage
  • 3.75% on a 30 year fixed, only about two years in
  • If we threw everything we could at the mortgage, including dividends from taxable accounts (while still maxing out our tax-advantaged accounts, because, taxes) we could put $40k additionally towards principal each year, but wouldn't be mortgage free until a little over five years. 
But because I want to be done by forty, here's our current plan, to be executed in July.
  1. Take our available cash and earmark that for the mortgage payoff. (We'd been dollar cost averaging these funds from two somewhat recent investment property sales.)
  2. Sell the most advantageous lots in our taxable accounts to minimize taxable gains. This means selling any lot with a loss first, then sell those with the smallest gains, looking for the lowest % long term gains (net, after offsetting losses, are taxed at 15%), trying to avoid any lots with short term gains (net, after offsetting losses, are taxed at our marginal tax rate of 22%).
  3. On the plus side, we'll have another $1,300 to invest each month without the P&I portion of the mortgage. If we spend at our same levels but invest that extra money, with our new, no-mortgage budget for two years, we'll only need about a 3% return each year to hit our goal of being financially independent by the last day I'm forty. Seems possible.
Okay, that's a lot of numbers. Way too many numbers for a Monday morning. Let's look at some cute golden retriever puppies to get our minds right.

Whew! Tiny goldens to the rescue.

Now that we're centered again, we can dig into the plan a bit more.

Since we are selling investments out of our taxable accounts (roughly $200,000 worth) this will generate some capital gains taxes.

Wait, where are you going? Please, don't leave. I'll keep it short. Here, here are more puppies.

Bottom line: we'll pay somewhere around $1,400 in capital gains taxes to execute this plan.

On the plus side, we'll avoid over $10,000 in interest each year without the mortgage. (This naturally would reduce as we move through the amortization schedule, but this is the amount we paid last year.)

I can already hear the personal finance blogger refrain though: what about the opportunity costs? Think of the gains I'll be missing out on.

And I'm usually the one telling other people to think about opportunity costs and to minimize taxes.

But we are very close to financial independence. We think a different strategy may be warranted.

We're planning on hitting financial independence, and likely leaving full time work, in roughly two years. At this point in the game, we're much more concerned with limiting losses than we are with maximizing gains. Seeing how our goal is a paid off home and a nest egg sustaining our spending, it's really a question of how we get there in the most prudent way. And we figure taking some money off the table to pay off the mortgage, and then building back up those funds, is a more conservative path forward.

The way we see it, there are three things that can happen over the next two years.

Scenario 1: the market is lower two years from now than it is today
In this scenario, we probably feel smart for taking some money out of the market before the correction. It's market timing, for sure. While we'll still have plenty invested, we will have limited our losses by taking some money out of play. Additionally, with the extra cash flow from having no mortgage payment, we'll be buying more stocks on sale, while the markets are down, than we would otherwise. And we'd certainly be buying more than if we'd been doing a traditional mortgage payoff, with all our available cash going towards principal payments.

This is probably the 'we got lucky and ended up doing a good thing' scenario, if it comes to pass.

Scenario 2: the market stays more or less flat over the next two years
This isn't the worst outcome, because even though we'd like to see some positive returns, we're at least getting a 3.75% by paying off the debt. That is technically a little better a return that we'd see in a flat market, even if we consider the 2% dividends we'd have received had we kept those funds invested.

Not a great scenario either way, but we'd probably still be ever so slightly better off paying down the mortgage.

Scenario 3: the market is higher two years from now than it is today
This is definitely the bad scenario for us, as we'd suffer opportunity costs for taking money out of the market. It's also probably the most likely scenario: most years, the stock market goes up.

Our Simpleton's Portfolio (sometimes called the No-Brainer Portfolio) has provided a median "real" return of 6.5% historically: a nominal return of 8.5% if we assume 2% inflation. So taking $260,000 out of our portfolio to avoid only a 3.75% interest rate would suffer some real opportunity costs: missing out on an additional 4.75% of returns: so over $12,000 in opportunity costs in an average year.

Still, in this scenario where we suffer these opportunity costs, the remainder of our nest egg we have invested is still going up. We're still seeing gains that are sufficient to make our goal of retiring by forty. In this bad scenario, while we could have reached our goal a few months earlier, we still reach financial independence in the timeframe we set for ourselves.

If that's the bad scenario, sign me up.

The bottom line is that we're trying to achieve a specific goal: not maximize our hypothetical gains. Because risk and reward go hand in hand. 

Avoiding opportunity costs is not my primary goal at this point in the process. Rather, I want to mitigate our risks. I want to minimize potential losses at this stage in our financial independence journey: if there's going to be a market correction, we want to ease its impact.

The way I see it, we're in the fourth quarter. We're up by four, there are just two minutes left in the game, and we have the ball on the fifty. My goal isn't to score another touchdown and run up the score. It's to protect the ball, make sure the team doesn't do anything stupid, run out the clock, and win the damn game.

I really couldn't care less if someone on the sidelines criticizes us for only winning by four, pointing out that we could have won by eleven. All I want is the "W".

As always, thanks for reading.

Disclaimers and FAQ
  • Just like everything on this blog, nothing in this post should be read as advice. I'm no professional, and you should consult with one if you're thinking about doing anything like this.
  • Why not just keep this very low interest mortgage all the way into your retirement, and use it as a potential arbitrage play? The short answer is that mortgages are bad for early retirees because they increase sequence of return risks. But read Big ERN's post for a full explanation.
  • Have you considered just continuing to invest, hopefully getting better than 3.75% returns, and then paying off the mortgage all at once? That was our first plan: the original mortgage swoop. Most of the time, this would get us to our goal a few months faster since usually the market returns are better than 3.75%. But if we get unlucky there's a correction in the next couple years, we'd suffer more losses. So we're trying to manage that risk rather than maximize our gains.
  • Why not just pay extra to principal with your extra cash each month? The timeline is just a bit too long: it would take us over five years with that plan, even if we threw everything we could at the mortgage...and I'd like to be done with full time work in about two. So we'd rather sell some investments rather than just relying on our paycheck income.


  1. I see the allure of paying off the mortgage and getting rid of the monthly obligation. It's a personal decision, not just financial. We have a low-rate mortgage (3%), we are 40's and retired from our corporate jobs (though we still earn through part-time consulting), and we keep our mortgage. We don't have excess money to pay it off, so the decision is hypothetical. But if we did have the principal balance available, I would set it aside in savings -- Wealthfront cash account is paying 2.5% these days. Once you pay the money back to the bank, it's hard to get it out, especially after you don't have a traditional job.

    1. Hi Caroline! I think you're in good company with that strategy: I believe Carl & Mindy at 1500 Days also keep a mortgage in early retirement. And at 3%, I can see the appeal.

      I hear what you're saying about illiquidity once the money is with the bank. I like to think we're still plenty liquid since paying down the mortgage still only takes a fraction of our nest egg. Once we take it out of the market, our thinking is that we don't really want to put it in savings (only 2.1% today with our Ally account, though it sounds like 2.5% is possible) if the alternative is getting 3.75%

      I suppose if interest rates on savings accounts got up to 4%, we'd come out ahead though. Hard to know where they're going!

  2. I'm pretty sure you won't regret paying off the mortgage. The only real "bad" scenario is if we see a repeat of the 2007 housing market crash. I don't think that's going to happen.

    1. Hi Joe!

      True, if we end up taking a big loss on real estate I guess we might do better having the bank with some skin in the game.

      The way I talked myself out of that was thinking that we wouldn't qualify for a short sale even if we were underwater. The bank would look at our assets and be like, "No, Brian, you are paying us for the whole loan amount."

  3. You know after our discussions here I am all for this plan. If we were ever to fully retire early, I would absolutely want to have our mortgage paid off first. And really, 2 out of 3 odds for making it the right financial decision as well is just icing on the cake.

    1. Thanks for your help with thinking this through, too, friend.

      I think the most likely scenario is that the 'bad' option comes through, and the market goes up over this two year period, and we kick ourselves a little for missing out on some gains.

      But if the market going UP is our 'bad' scenario, then that's the kind of hedge I want to make, you know?

  4. Your bad scenario is really not so bad! Considering you're not wiping out your holdings to make this happen AND you are freeing up cash flow which you can then plow right back into the market anyway if you want to it seems to make sense to sell while the market is high. Sure that's market timing talking but we'd be saying the opposite if it was down, wouldn't we? It's all market timing in the moment. Like when I regretted buying a load of VTSAX at $71 a share a month ago and now am kind of glad I did because it keeps going up. Who the heck knows what's going to happen in two years? No one, so we might as well do the best we can with the knowledge we have.

    I keep going back and forth and right now as desperately as I want to pay off that mortgage, we simply don't have enough in our brokerages to eliminate the mortgage AND still have enough capital to grow so I sit here, bathing myself in exaggerated patience.

    1. Yeah, that's how we feel about it, Revanche. We know this is at least partially market timing but, as we get close to the goal our strategies seem to have shifted.

      I think with a bigger mortgage we might have to make more of an 'all or nothing' scenario, so I hear where you're coming from. For what it's worth, I think your patience sounds like a good strategy: it's hard to take all your money off the table, just in case the market continues to go up.

  5. Well seems like a dumb move to me. Good thing you always change your mind though.

  6. I get it. I'm behind on retirement, but I'm still diverting a little money (like the guest house rent) to go toward an early mortgage payoff. It's not the most mathematically sound, but anxiety is a thing and competition is high in my company's industry. I want the house paid for just in case the worst happens.

    1. I tend to the think the math can work out a lot better than people think. Most people in favor of an arbitrage play plug in 7 or 8% into their little calculator, not considering someone's individual circumstances (i.e. - are you planning on retiring in 2 years, because as Big ERN points out, taking a mortgage into early retirement is not smart)? What is the current CAPE? And like you said, what's your work situation look like?

      The time you're investing this money is pretty important context.

  7. I don't get why not to stop at step 2 and then put any new money into the mortgage instead of directing it towards taxable. At that point the interest saved should be much lower. It doesn't have to be all or nothing, and as you've noted it isn't really a 3.75%/year savings every year. That 10K number saved in interest dwindles down to not much at all. (If you click on our mortgage tag you can see that monthly interest saved drop to piddly amounts for a long time period.)

    I mean, sure it's emotional and stuff, but honestly we've had our mortgage paid off for a few years and I would not have taken a tax hit to do it. We slowed down pre-paying once the interest saved wasn't worth much and directed the money we had been using to prepay towards 529 instead. Really what paying off our mortgage (and concurrent ending daycare payments) has encouraged is additional spending. We don't get enough extra each month to make it worthwhile to pay transaction costs to the stock market, so it sits in savings for a while and then we buy fancy cars and renovate kitchens. Which some might argue was overdue. But it was a lot easier to direct a couple extra thousand each month towards mortgage prepayment than to buying bonds in taxable. (Btw, my taxable index funds drip like crazy, so I'm not sure I agree with the Bogleheads advice about capital gains... and I'm fairly sure I've also read the opposite advice on Bogleheads re: where bonds should be in a large portfolio when I was doing research on a post on the topic. Sure, if you're investing in just growth stocks that don't spit dividends, but that's not a diversified portfolio.)

    You do you. It's not like it's the worst thing in the world to have a paid off mortgage even if you are giving excess money to Trump's regime I mean capital gains tax in the process.

    1. Whew, a lot to unpack there.

      "I don't get why not to stop at step 2 and then put any new money into the mortgage instead of directing it towards taxable. At that point the interest saved should be much lower."

      I feel like the three prior posts on mortgage pay off do a pretty good job of explaining this, but the short answer is that it would take 5+ years to do this and we are retiring in two, so having a mortgage for those years would increase sequence of return risk, which is the very worst risk for an early retiree. See the Big ERN post linked in all three mortgage swoop posts for a more detailed explanation of why mortgages in early retirement are a bad idea.

      I think that's the point I'm not getting across: that I'm not all that worried about opportunity costs (we have enough money to be retiring around 40...who cares about ten grand in opportunity costs or a whopping $1400 in taxes), but I'm concerned about sequence risk. That's the thing that can easily derail our plans. Mortgages make sequence risk much worse.

      We had our prior mortgage paid off once before and didn't end up spending materially more, but maybe it'll be different this time around.

      Re: bonds, we keep those in tax advantaged accounts but, yeah, the dividends stock funds throw off are creating taxable income, too. Good problem to have, IMO.

      Don't really love the comment about giving excess money to Trump's regime but, like you said, you do you.

  8. I think the way you are paying off your mortgage is the way to do it. You are getting close to retiring early, and that freed up expense is a huge deal.

    Since we are a ways off from hitting early retirement (10-15 years), we are going to keep the mortgage and try to make as much gains as possible. In 10-years we might end up doing what you are doing here, and I think minimizing risk is a great decision with the reasons you stated.

    There is always going to be pros/cons with decisions like this, but I think the worst case scenario is not all that bad.

    With how everyone is anticipating a recession soon, I kind of think it might not actually happen, but we will see. For your case and my case (since I'm just starting to invest large money in the stock market), I hope a recession hits this year or next year. That way we can make more gains in the future. Good luck!

    1. Thanks, Chris. While I certainly don't expect everyone to want to pay off their mortgage, I think when you're less than 24 months from your planned retirement date, I think it's something to consider. I certainly wouldn't be telling a 63 year old with a paid off home to leverage up and get a mortgage: keeping a mortgage going into a retirement is substantially equal to taking one out prior to retirement (sunk costs and all that).

      In your situation, where you're 10-15 years out, I definitely see the argument to use low interest leverage to create a possible arbitrage. As you approach your retirement date, then moving to less risk in some form or fashion is perhaps a prudent move.

      Like you said, the worst scenario for us (#3) is not bad...that's kind of why we set it up this way.

      Like you, I'm not 100% we're due for a recession but just in case, I want to hedge. If it happens prior to our retirement, I guess there's a silver lining. Thanks for reading, Chris.

  9. You never talked to ME about this, but that's okay. I see how you roll... ;-)

    Seriously - great plan, my friend. We paid ours off earlier this year and it's just one less thing to worry about now. I always considered those payments our "bonds portfolio" -- low yield, but low volatile hedge against market uncertainty. Or something...

    1. Cubert!

      I should've reached out, as someone who's on a similar timeline to FI.

      Like you said, there's a negative bond aspect to having a mortgage. (Which spurned our bond mortgage swoop idea which we ultimately didn't go with. I like the idea of using bonds only to pay the mortgage but the problem is that if there's a market correction, you don't have bonds in your portfolio to rebalance. So we ended up just keeping our asset allocation so we're protected in case of a big correction: if stocks take a nosedive we can use the bonds to rebalance & make a big stock purchase once a year.)

  10. We paid our mortgage off in 6 years and have lived 14 years mortgage free. The peace of mind is better than any lost opportunity costs. Good for you!

    1. Thanks, friend! And congratulations to you as well: 14 years mortgage free is a pretty amazing thing.

      I'm definitely looking forward to some peace of mind, too: life is throwing us some stress lately, and I will take what I can get to mitigate that. :)