Knowing when to give up on legacy projects

The last week or so I've been trying to get an ancient legacy Ruby on Rails app up to date. This was running Ruby 1.9.3 and rails 3.0.6 and the dependencies have not been touched at all in close to 10 years. This project led me to write the article about why I think executable file formats aren't the best way to track dependencies.

A crucial issue with a project like this is that nobody actually remembers any of the intentions of anything at the time the code was written. So if anything isn't documented or explicit in the code it becomes a times consuming active process to re-discover it.

Turns out that the packaging on this project was a mess. There's a gemfile but it appears that the production environment didn't have exactly the same things as described in the gemfile. Also recreating the production environment is close to impossible since this project never had anything like a build script or containerization. I don't think that's a shortcoming of the people who made this either, Docker came out a few years after this project started. Also back then I think people just didn't have as any good tools for dependency management either. One of the first challenges I ran into was that the version of Ruby was old enough that it was hard to find good docker images for Ruby 1.9.3.

So it turns out that the main issue with this is that the build step wasn't documented, this made it especially difficult to try to figure out exactly what was installed and what was not. Adding to this problem was the fact that some dependencies had been pulled and are now hard to install. Now I understand why this step tends to not be so good in older projects, it was simply far harder to make repeatable builds back then, additionally a lot of people just weren't aware of containerization.

If you are creating a Rails project now I highly recommend using something like Bundler, not having good dependency management is just hell. If there's one take away from this article it's that using the proper dependency management tools, in this case Bundler, saves an enormous amount of hassle.

I kept running into obscure installation issues like the following:

/var/lib/gems/2.3.0/gems/activesupport-3.0.6/lib/active_support/values/time_zone.rb:272: warning: circular argument reference - now
/var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:94:in `rescue in block (2 levels) in require': There was an error while trying to load the gem 'sorcery'. (Bundler::GemRequireError)
Gem Load Error is: uninitialized constant ActiveSupport::LogSubscriber
Backtrace for gem load error is:
/var/lib/gems/2.3.0/gems/activerecord-3.0.6/lib/active_record/log_subscriber.rb:2:in `<module:ActiveRecord>'
/var/lib/gems/2.3.0/gems/activerecord-3.0.6/lib/active_record/log_subscriber.rb:1:in `<top (required)>'
/var/lib/gems/2.3.0/gems/activerecord-3.0.6/lib/active_record/base.rb:24:in `<top (required)>'
/var/lib/gems/2.3.0/gems/sorcery-0.9.1/lib/sorcery.rb:73:in `<module:Sorcery>'
/var/lib/gems/2.3.0/gems/sorcery-0.9.1/lib/sorcery.rb:3:in `<top (required)>'
/var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:91:in `require'
/var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:91:in `block (2 levels) in require'
/var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:86:in `each'
/var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:86:in `block in require'
/var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:75:in `each'
/var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:75:in `require'
/var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler.rb:106:in `require'
/src/config/application.rb:8:in `<top (required)>'
/var/lib/gems/2.3.0/gems/railties-3.0.6/lib/rails/commands.rb:28:in `require'
/var/lib/gems/2.3.0/gems/railties-3.0.6/lib/rails/commands.rb:28:in `block in <top (required)>'
/var/lib/gems/2.3.0/gems/railties-3.0.6/lib/rails/commands.rb:27:in `tap'
/var/lib/gems/2.3.0/gems/railties-3.0.6/lib/rails/commands.rb:27:in `<top (required)>'
script/rails:6:in `require'
script/rails:6:in `<main>'
Bundler Error Backtrace:
    from /var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:90:in `block (2 levels) in require'
    from /var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:86:in `each'
    from /var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:86:in `block in require'
    from /var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:75:in `each'
    from /var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/runtime.rb:75:in `require'
    from /var/lib/gems/2.3.0/gems/bundler-1.13.7/lib/bundler.rb:106:in `require'
    from /src/config/application.rb:8:in `<top (required)>'
    from /var/lib/gems/2.3.0/gems/railties-3.0.6/lib/rails/commands.rb:28:in `require'
    from /var/lib/gems/2.3.0/gems/railties-3.0.6/lib/rails/commands.rb:28:in `block in <top (required)>'
    from /var/lib/gems/2.3.0/gems/railties-3.0.6/lib/rails/commands.rb:27:in `tap'
    from /var/lib/gems/2.3.0/gems/railties-3.0.6/lib/rails/commands.rb:27:in `<top (required)>'
    from script/rails:6:in `require'
    from script/rails:6:in `<main>'

This is really strange because Rails should install ActiveSupport but yet something hasn't gone right. Searching around for solutions isn't that easy for these problems because the error output is especially hard to search for.

Generally speaking when I run into difficult problems I like to dig into the details a bit more, so I found this post that describes the way in which Gems work and this post about how Rails loads dependencies. My heart sank when I read the following quote from the article:

Most of the time, gems in Ruby Just Work. But there’s a big problem with Ruby magic: when things go wrong, it’s hard to find out why.

You won’t often run into problems with your gems. But when you do, Google is surprisingly unhelpful. The error messages are generic, so they could have one of many different causes. And if you don’t understand how gems actually work with Ruby, you’re going to have a tough time debugging these problems on your own.

This really seemed spot on to me, I'm good at searching up technical problems and I was finding very little help. It was obvious that the cost in terms of time and effort for solving this would be high so I was immediately left with a choice here, I could try to keep plowing on or I could cut my losses. The 60 commits I'd made cast a shadow of the sunk cost fallacy over the whole thing. But now that I'm more mature I realize that sometimes you just have to cut your losses, there's no point in throwing away more time just because you've already thrown away a bunch of time but emotionally and organizationally this can be a tough sell even if you know it's the right choice. I asked what the intentions were with the dependencies and got a "I don't know it was 10 years ago" response and any remaining doubts I had were removed, a change in approach was needed.

The moral of the story is that you really need to document your dependencies preferably in some form of well structured file format that can be used with tooling. In this particular instance there were dependencies for the code that were not present in the Gemfile and nobody really knew what exactly was installed previously. Trying to fix this incrementally might just not be possible, at least not in any sort of cost effective manner. Ever since Joel Spolsky two decades ago wrote about the disaster that was the Netscape re-write a lot of software engineers have been hesitant to engage in large scale re-writes. Part of his reasoning is that there's a lot of value in working code as it embodies business value. In this case nothing worked and it didn't appear to be easy to actually get anything to work in a manner that maintained the integrity of the old code, meaning that less value was going to be lost via the re-write compared to a working product. An incremental improvement would just be getting the app into a working-but-still-legacy state and the effort might have been bigger than a re-write.