Monday, March 18, 2019

Our Mortgage Swoop

Our Mortgage Swoop
Ever since we purchased our first home, Mrs. Done by Forty and I have had no idea what to do with our mortgage. When we first got in to personal finance we were against debt of any sort, so we threw every dime we could at our first mortgage, paying it off in just over three years.

Seeing how we did this from 2010-2013, this plan had opportunity costs that are so large that it hurts my heart to even think about it. Still, it felt good at the time.

After paying off the house and enjoying debt-free bliss for a year, we up and bought a couple adorable rental properties, complete with tiny mortgages.

After a while, we realized that opportunity costs are just as real as any other cost. So we took out another mortgage on our primary residence in 2016, just three years after paying it off. I always wished I'd called in to Dave Ramsey's show to tell him that.

Now, nine years after buying our first home, both of the rentals and our first primary residence are long gone, as are our dreams of lording over a real estate empire. It turns out we hate being landlords.

Do we hate mortgages, too? Let's find out, because we're about to pivot on our mortgage once again.
Every so often I'll speak with a CFP who offers a free financially planning call, which is really just a pitch to start doing business with them. Vanguard has a call like this once a year, and one of the hidden benefits in our company's EAP plan is a call with a CFP every twelve months, too.

Each time I ask them the same thing: if we're planning on retiring early, should we pay off the mortgage? I send them Big ERN's post ahead of time, the one on how mortgages and early retirement don't mix, so they can see where the question is coming from and why the answer might be different for someone retiring at 40 instead of 65. But these CFPs must be busy or lazy because I don't think a single one has read it yet.

Somewhat surprisingly, they all say to keep the mortgage around, even with our plan for a fifty or sixty year retirement, even with our protests about sequence of return risk (the bad thing that happens if you get some very bad returns in your first years of retirement). These dudes all seem more concerned with us having bigger lump sums when I'm in my eighties than they are with the sequence risks in the first five to ten years of an early retirement.

Maybe I'm just an asshole, but I don't think any of these guys know what they're talking about when it comes to the risks inherent with a fifty year retirement. The only financial planner I really trust, Joe Saul-Sehy, was nice enough to answer this question on one of his shows: he thought paying off a mortgage prior to an early retirement was prudent, and he gave the green light on our latest plan to do so: the Mortgage Swoop.

Rather than making big payments to principle over a period of years, our new plan is simply to wait until our nest egg gets to $1M + our mortgage principle balance (currently $260,000, at 3.75% on a 30 year fixed), and then sell $260k of investments and pay the whole thing off at once, leaving just the million.

Like, we come in all quick...and then, swoop, no more mortgage.

This isn't coming off well in print, but in real life I make a swoop gesture when Mrs. Done by Forty and I talk about it. You know, a swoop.

Our Mortgage Swoop

Also, swoop sounds like other fun words.

What we like about this plan is that it gets us to be mortgage free and mitigates the dreaded sequence of return risk, but it does so in a way that protects us in case something bad happens in the middle of a traditional mortgage pay off plan (where we'd just pay extra to principle every month). In a traditional mortgage pay off plan, there's some risk that you could lose your job or have a costly emergency midway through your mortgage payoff. If that happens in the middle of our plan, we can adjust it, because all the money's just sitting there in a brokerage account.

And since we'd wait until we have a million plus our mortgage balance, we'd be left with enough to generate $40,000 annually under the 4% rule, or $35k annually assuming a 3.5% safe withdrawal rate.

This admittedly doesn't sound like nearly enough money, but there's the fact that the principle and interest portion of our mortgage payment accounts for over $15k in our annual spending. If we can make it on a $50,000 budget with the mortgage, then our annual expenses would immediately drop down below $35k once the mortgage is paid off.

(A quick aside: it bugs me when bloggers say they spend something like $24k annually but don't point out that their mortgage is paid off, nor do they state what their budget was when they still had a mortgage payment. The latter is much, much more representative to just about anyone reading your blog. Use that figure, dudes, or be sure to note that your tiny-but-good-sounding budget is misleading because it just happens to miss the largest line item in everyone else's budget.)

Anyway, we don't really know exactly when we'll hit this $1.26M figure, being in the tenth year of a bull run and all. But my optimistic guess would be sometime in early 2021, roughly two years from now.

If we were to pull the trigger on our mortgage swoop, we'd technically be financially independent at that point, presuming our budget doesn't keep growing uncontrollably. We might even be able to time our early retirement to coincide with the birth of our second baby, whom we're tentatively naming "MC Baby" for the blog.

If the timing worked out, I could stay home and help Mrs. Done by Forty post-birth more than I did with Baby AF, when I only had five weeks of parental leave and vacation before having to head back to work.

Anyway, that's the loose plan: hopefully pull off the mortgage swoop sometime in early 2021, hit financial independence, bring MC Baby into the world later that year, and maybe pull the trigger on early retirement that year as well.

Some readers may have already noticed the big problem with this plan already: won't there be a mess of capital gains taxes to pay if you sell $260,000 of investments all at once?

Potentially, yeah. Without knowing what the market is going to do in the next few years I can't say what kind of gains we'd be looking at. Still, we have two half-baked plans to mitigate those taxes.

One, we plan on selling the lots that would incur the smallest amount of gains (probably the most recent deposits into the brokerage account) and trying to do some tax loss harvesting to offset the gains as well.

Our second tactic is to try to time my early retirement to hit in the same year as the mortgage swoop, so we can potentially get many or all of those capital gains taxed at 0%. Let me explain. If we retire early enough in that year that our taxable income is under $78,850 (or whatever the cap will be in 2021), then we'll have some space to have those gains taxed at 0% instead of 15%. [There's a good primer on how to leverage the 0% capital gains rate here.]

With some lucky timing and some decent planning, it's possible there might not be many, or any, capital gains taxes to pay. We won't let the tax tail wag the dog, either. If we're not actually ready to pull the trigger on early retirement (e.g. - if there's a market correction in 2021) then I'm not going to quit my job just to avoid a few thousand in taxes.

We've changed our minds on our mortgage plans many, many times over the years so who knows if this is the plan we'll actually stick with, let alone if it's a good idea. But it's an idea Mrs. Done by Forty and I are both excited about, and that hardly ever happens when it comes to our finances.

But I'd love to hear what you readers think about it. Is this a good idea, or is there a better way?

*Photo is from Ruth and Dave at Flickr Creative Commons.


  1. I started paying in chunks last year on the mortgage. Biggest reason being (intended or not, I know you lean left hehe) the new tax plan has us taking the standard deduction. So no benefit to having a mortgage anymore tax wise for us. So the real return of the mortgage % plus the piece of mind when it's gone, seems to make more sense to me. I think some day I do let that tax tail wag the dog but so far it's working out well for me. We max out pre-tax and now with QBI deduction we can get pretty creative between me and the Mrs.
    I also see some crazy low yearly budgets and thing that is only possible with a paid off mortgage. I will be curious when we have no mortgage and kids move on how low the yearly # could get, but it won't be 24k either way I just don't get it. I would be bored if we didn't actually do stuff like eat out and go places etc.

    1. Hi Chris! We bought our new, big house in 2017 so I, too, know the awkwardness of having a plan where you were going to get to itemize your mortgage interest and then having the rug pulled out. I've never, once, gotten to itemize my interest on our primary residences. Go figure.

      When we paid off our first mortgage, it sounds like we used the same plan you are. It works! In retrospect we realized there's some small risk in losing your job mid-way, but thta might not be as much of a concern now that we have a larger nest egg than we did in 2010-2013.

      And yeah, we're not going to be down near $24k, either. ;)

  2. Should there be congratulations in order, or is MC Baby still a gleam in your eyes?

    1. Whoops, got you mixed up with Rby40.

      Re: mortgage payments-- it makes more sense to do early payments than late ones (early ones make a bigger impact). Swooping makes no sense. Just keep paying, or pay off early.

    2. Hey there nicole & maggie.

      MC Baby is still a gleam, but thanks for the kind words!

      One thing that might make a little bit of sense is that we're investing the funds between now and the mortgage swoop. And while no one has a crystal ball, on most years those investments beat a 3.75% return.

      So maybe there's just a little method to our madness. ;)

  3. I'm against the swoop. Why not do it over a few years instead? That way you can spread out the capital gains. It worked out pretty well the last time you did it, right? Why change?
    Well, if you can get to 0%, then it'd be worth it. Maybe spread it over 2 years?

    1. Hey there, Joe.

      Hopefully this came across in the post but one thing we're not keen on with a traditional mortgage pay off is that you can potentially be laid off in the middle of the plan, with no way of accessing that money.

      I see what you're saying about spreading out the capital gains but if we coincide the swoop in the same year that we retire early, we could lock in a 0% tax rate on some, or perhaps all, of the gains.

      We could spread out the gains over a couple of years like you said. But it seems we're going to be in the 15% cap gain bracket no matter what we do while I'm still working: is there an advantage of splitting that over two years? Or are you saying split it over two years after we retire?

  4. I say do whatever makes you the most comfortable. If you prefer to swoop (which is just a fun word, and yes I do have "Shoop" running through my head now so thanks for that), then swoop. I personally would probably just pay off aggressively now, but I'm in a very different situation than you are. I want to guard against any future job loss, which isn't likely but is possible. So I want my mortgage gone, just in case. You're already planning for a job loss -- in a good way -- so your risk tolerance is going to be higher than mine.

    1. Hi Abigail,

      Totally. Knowing and planning for a job loss is way different than protecting against an unknown one. I personally feel the swoop is actually less risky in the event of any job loss (you'd have a big pile of money you can access in the event of a job loss midway through the plan, rather than having paid that money in advance principle payments). However, we did do the traditional mortgage payoff plan before and it worked out fine. Maybe I'm overstating the risk (which is normal for me).

  5. Hopefully you meant OG, because I'm not a planner! Now I just play one on the "internet radio" thingy. ;-) As you already know, the swoop isn't my favorite plan, although most wealthy people pay off their debt quickly. I think the risk of a sequence of returns issue when your Monte Carlo % probability of success is high becomes much less. I still then like the interest rate arbitrage game. Even better: how much money do you need to set aside to make the payments to the mortgage so that you don't have to worry about it? BUT none of this is relevant if you have enough money to pay down the mortgage and still retire successfully and stay retired. Pay down the debt, don't worry about it ever again. No mess, no fuss. Clean plan. We'd usually play with mortgage scenarios for people that HAD to have a little risk in their plan because they BARELY had enough to make it.

    1. Hey now. You said on the radio thingy that you LIKED the plan. You wait until I write a whole post to say you don't it? I'm telling your mom.

      Seriously, I could have sworn you signed off on the plan maybe a year ago on the show and that it was OG who said to leave the mortgage around. Maybe I'll go try to find the episode.

      We could leave the same amount as our principle invested rather than paying it off, and that figure could likely safely "pay" the mortgage on an ongoing basis simply because the interest rate on the loan (3.75%) is very close to our safe withdrawal rates on our investments (3.5% or 4% depending on who you talk to).

      However, given that the investment's SWR rate is basically the same as the interest rate, I don't think I'm really coming out that far ahead with the arbitrage.

      I'm personally WAY more concerned about sequence of return risks than the opportunity costs.

  6. I'm firmly in the "don't pay it off" camp. It's lonely over here. Not many fellow campers...

    What we did: We're at 3.25% on a 15 year with about 9 left to go. Very simply, I just think I'll have more money in the long-term with the mortgage and my money invested than I would have otherwise. Over the course of my loan, I'll pay $37,000 in interest. Since we bought the home (June of 2013), the S&P has returned 91.226% (with dividend reinvestment). We could have paid cash, but we invested the $140,800 instead. So, in the S&P 500, that money has now appreciated $128,446. I'm way ahead. Of course there are caveats:

    1) I put my money into the markets at a really, really good time. Still, I'm guessing that the markets will beat 3.75% over the next 20 years.
    2) If I would have paid cash for the house, I could have been investing an extra $1,000+ into the markets every month. However, I'm still ahead by taking the "lump sum at the beginning" route.

    But, personal finance is personal. What floats my boat may sink yours. Whatever helps you sleep at night.

    Question: Say interest rates keep going up and you can get 3.75% on a bank account. Would you still pay the mortgage off? To me (with very little thinking), a better strategy may be to sell off enough investments to cover spending for a year or two, put the money in the savings account, and keep the mortgage. You have cash then to ride out the bad period, but most of your money keeps working for you.

    1. Hi Carl! For what it's worth, I think there are quite a few very smart people in your camp. And at 3.25 on only a 15 year, that's pretty sweet.

      To answer your question, the arbitrage play to get 3.75% or more in a bank account has certainly crossed our minds. But I don't think it's likely in the timeframe we're talking about hitting FI in, and this post from Big ERN about holding a mortgage in RE has us spooked. We find his stuff very convincing.