Skip to content
PLAY VIDEO PLAY VIDEO PLAY VIDEO
By Mikel Lindsaar

Bundler, GitHub, gemspecs, dependencies and versioning

This post could also be entitled "Oh Espresso machine, I love thee".

StillAlive is a reInteractive application that monitors that websites are operating as expected - not just making sure the front page comes up, but by using customer written test scripts that navigate around and log into their web applications every minute and alerting on downtime and errors.

Over the weekend, we upgraded our service to use Ruby 1.9.3 to take advantage of some new features and fixes in the ruby Net library. We did the upgrade, fixed up three gems that needed to be changed to work with 1.9.3 and ran our test suite. Happily, all 3000+ specs and steps passed. We then did a dummy upgrade with a recent restore of the production database, again all looking good, and finally deployed to staging, again, everything 100% OK.

This was looking to be a very simple upgrade!

However, after deploying to production, a couple of our clients reported that some of their recipes were failing due to an encoding related problem. Happily, after a couple of hours of hunting it down, I found that the Mechanize library had already patched this, but was not released as a gem version yet.

Bundler usually makes this trivial to solve, first you fork the Mechanize library to your own repository (see Bundler and Public Git Sources) and then you just put something like this in your Gemfile:

# Fixing gemfile against fork until commit 1561fae6bd458e42a10d41e5a6a728b253aafe96 is
# rolled into Mechanize, probably at the 2.1.1 release, once we are at 2.1.1 revert back to the Gem
gem     "mechanize", :git => 'git://github.com/reInteractive/mechanize.git',
        :ref => 'f10d647e1ff653a11433a24e965b855b2ee49fed'

Note, when I am depending on a git reference as opposed to a published gem, I always comment my Gemfile. In 5 months it is good to know exactly why you were depending on a git source instead of the gem and know for sure if it is OK to revert.

However, in the case of Mechanize, there is no mechanize.gemspec checked into the git repository as it is autogenerated by their Rakefile. As my application depends on a version of Mechanize that is at least version 2.1.0, Bundler then complains that it can't find a version of Mechanize at the git repository:

Could not find gem 'mechanize (>= 0) ruby' in git://github.com/reInteractive/mechanize.git (at f10d647).
Source does not contain any versions of 'mechanize (>= 0) ruby'

The excellent Bundler docs explain how to solve this:

If a git repository does have a .gemspec for the gem you attached it to, a version specifier, if provided, means that the git repository is only valid if the .gemspec specifies a version matching the version specifier. If not, bundler will print a warning. If a git repository does not have a .gemspec for the gem you attached it to, a version specifier MUST be provided. Bundler will use this version in the simple .gemspec it creates.

So then the Gemfile line becomes:

# Fixing gemfile against fork until commit 1561fae6bd458e42a10d41e5a6a728b253aafe96 is
# rolled into Mechanize, probably at the 2.1.1 release, once we are at 2.1.1 revert back to the Gem
gem     "mechanize", "2.1.0", :git => 'git://github.com/reInteractive/mechanize.git',
        :ref => 'f10d647e1ff653a11433a24e965b855b2ee49fed'

But this won't work by itself, because gemspecs also list dependencies which if present, Bundler will automatically require. Opening up the Rakefile in the Mechanize repository, I see:

  self.extra_deps << ['net-http-digest_auth', '~> 1.1', '>= 1.1.1']
  self.extra_deps << ['net-http-persistent',  '~> 2.3', '>= 2.3.2']
  self.extra_deps << ['nokogiri',             '~> 1.4']
  self.extra_deps << ['ntlm-http',            '~> 0.1', '>= 0.1.1']
  self.extra_deps << ['webrobots',            '~> 0.0', '>= 0.0.9']
  self.extra_deps << ['domain_name',          '~> 0.5', '>= 0.5.1']

So we need to add these to the Gemfile as well:

# These next 6 gems (nokogiri, net-http-digest_auth, net-http-persistent, ntlm-http, webrobots, domain_name)
# are needed to be explicitly required because we are creating a dummy gemspec with mechanzie
# off the github repository.
gem 'nokogiri',             '~> 1.5.0'
gem 'net-http-digest_auth', '~> 1.1.1'
gem 'net-http-persistent',  '~> 2.3.2'
gem 'ntlm-http',            '~> 0.1.1'
gem 'webrobots',            '~> 0.0.9'
gem 'domain_name',          '~> 0.5.1'

# Fixing gemfile against fork until commit 1561fae6bd458e42a10d41e5a6a728b253aafe96 is
# rolled into Mechanize, probably at the 2.1.1 release, once we are at 2.1.1 revert back to the Gem.
# Also remove the 5 dependent gems above
gem     "mechanize", "2.1.0", :git => 'git://github.com/reInteractive/mechanize.git',
        :ref => 'f10d647e1ff653a11433a24e965b855b2ee49fed'

Finally! Now doing a bundle install actually worked with no complaints.

Running the specs though had a bunch of errors coming up. Tracing this down, I found that Mechanize was being required explicitly in my code, and now that mechanize was also in the Gemfile, bundler was also requiring it. In my case this was causing a problem. Bundler provides a simple way to solve this by adding :require => false making the final Gemfile entry look like this:

# These next 6 gems (nokogiri, net-http-digest_auth, net-http-persistent, ntlm-http, webrobots, domain_name)
# are needed to be explicitly required because we are creating a dummy gemspec with mechanzie
# off the github repository.
gem 'nokogiri',             '~> 1.5.0'
gem 'net-http-digest_auth', '~> 1.1.1'
gem 'net-http-persistent',  '~> 2.3.2'
gem 'ntlm-http',            '~> 0.1.1'
gem 'webrobots',            '~> 0.0.9'
gem 'domain_name',          '~> 0.5.1'

# Fixing gemfile against fork until commit 1561fae6bd458e42a10d41e5a6a728b253aafe96 is
# rolled into Mechanize, probably at the 2.1.1 release, once we are at 2.1.1 revert back to the Gem.
# Also remove the 5 dependent gems above
gem     "mechanize", "2.1.0", :require => false,
        :git => 'git://github.com/reInteractive/mechanize.git',
        :ref => 'f10d647e1ff653a11433a24e965b855b2ee49fed'

So there you have it. Pushed into production, this solved the issues for our clients. Everyone is happy... even my espresso machine.

Latest Articles by Our Team

Our expert team of designers and developers love what the do and enjoy sharing their knowledge with the world.

We Hire Only the Best

reinteractive is Australia’s largest dedicated Ruby on Rails development company. We don’t cut corners and we know what we are doing.

We are an organisation made up of amazing individuals and we take pride in our team. We are 100% remote work enabling us to choose the best talent no matter which part of the country they live in. reinteractive is dedicated to making it a great place for any developer to work.

Free Community Workshops

We created the Ruby on Rails InstallFest and Ruby on Rails Development Hub to help introduce new people to software development and to help existing developers hone their skills. These workshops provide invaluable mentorship to train developers, addressing key skills shortages in the industry. Software development is a great career choice for all ages and these events help you get started and skilled up.

  • Webinars

    Webinars

    Webinars are our online portal for tips, tricks and lessons learned in everything we do. Make the most of this free resource to help you become a better developer.

    Learn more about webinars

  • Installfest

    Installfest

    The Ruby on Rails Installfest includes a full setup of your development environment and step-by-step instructions on how to build your first app hosted on Heroku. Over 1,800 attendees to date and counting.

    Learn more about Installfest

  • Development Hub

    Development Hub

    The Ruby on Rails Development Hub is a monthly event where you will get the chance to spend time with our team and others in the community to improve and hone your Ruby on Rails skills.

    Learn more about Development Hub

Get the “reinteractive Review” Monthly Email