By Evan Sangaline | September 6, 2017
NOTE: Be sure to check out Running Selenium with Headless Chrome if you’re interested in using Selenium in Python instead of Ruby.
Since Google added support to run Chrome and Chromium in headless mode as of version 59, it has become a popular choice for both testing and web scraping. There are a few Chrome-specific automation solutions out there, such as Puppeteer and Chrome Remote Interface, but Selenium remains a popular choice due to it’s uniform API across web browsers and it’s support for multiple programming languages. In this tutorial, I’ll walk you through the process of getting started working with Selenium and headless Chrome in Ruby.
Installing the Prerequisites
Before using Selenium with headless Chrome in Ruby, you will need to install a few dependencies. You’ll of course need to install Ruby and Google Chrome if you haven’t already done so, but chances are that you have these already if you’re looking for instructions on how to use Selenium with headless Chrome in Ruby ☺.
You’ll also need to install ChromeDriver which will allow Selenium to communicate with your headless Chrome instance.
This can be done on Linux and macOS (OS X) by running the following code in a terminal (with
PLATFORM set to the appropriate value for your system).
# platform options: linux32, linux64, mac64, win32 PLATFORM=linux64 # the directory to install the chromedriver binary in INSTALLATION_DIRECTORY=~/bin/ # create it if necessary mkdir -p $INSTALLATION_DIRECTORY # and add it to PATH export PATH="$INSTALLATION_DIRECTORY:$PATH" # and add the PATH modification to ~/.bashrc echo 'export PATH="$INSTALLATION_DIRECTORY:$PATH"' >> ~/.bashrc # automatically find the latest version VERSION=$(curl http://chromedriver.storage.googleapis.com/LATEST_RELEASE) # download and extract the latest version curl http://chromedriver.storage.googleapis.com/$VERSION/chromedriver_$PLATFORM.zip \ | bsdtar -xvf - -C $INSTALLATION_DIRECTORY
This code will create a local directory called
bin in your home directory and then extract the
chromedriver executable there.
You can choose a different directory by setting
INSTALLATION_DIRECTORY to a different value.
If you choose
/usr/local/bin/ or any other system directory then you will probably need to run the last command as the root user due to permissions.
Also note that the installation directory is added to your
PATH variable so that
chromedriver can be run.
If you’re familiar with
PATH then you might want to manage these changes yourself, but the commands should work as written.
After installing ChromeDriver, you should check that it’s installed properly by running
chromedriver in the terminal.
If you see something like
Starting ChromeDriver 2.32.498513 (2c63aa53b2c658de596ed550eb5267ec5967b351) on port 9515 Only local connections are allowed.
then it means that the installation was successful and you can close the ChromeDriver instance by pressing
ctrl-C in the terminal.
If you see a
bash: chromedriver: command not found error instead then it either means that the earlier download failed or that you’re in a terminal without the updated
You can trying running
source ~/.bashrc to update
PATH, but if that doesn’t work then you should try running the earlier commands again.
The final piece of software that you’ll need to install is the
selenium-webdriver gem for Ruby.
This can be installed by running
gem install selenium-webdriver
as long as you have RubyGems installed. Once that installation process is complete, you’re ready to actually use ChromeDriver with Selenium to automate controlling headless Chrome!
Running Selenium with Headless Chrome in Ruby
Configuring Selenium to use headless Chrome is actually pretty straightforward once you have all of the dependencies properly installed.
For example, putting the following code in a file called
require "selenium-webdriver" # configure the driver to run in headless mode options = Selenium::WebDriver::Chrome::Options.new options.add_argument('--headless') driver = Selenium::WebDriver.for :chrome, options: options # navigate to a really super awesome blog driver.navigate.to "https://intoli.com/blog/" # resize the window and take a screenshot driver.manage.window.resize_to(800, 800) driver.save_screenshot "intoli-screenshot.png"
and running it with
ruby take-screenshot.rb will launch Chrome in headless mode, navigate to the Intoli blog, and save a screenshot to
The screenshot should look something like this and show you our most recent article.
As far as headless mode is concerned, the key part of the above code is the use of the
Selenium::WebDriver::Chrome::Option object and the specification of the
The actual navigation, resizing the window, and taking a screenshot are all part of the standard Selenium Ruby API and work the same regardless of whether or not Chrome is running in headless mode.
Once you’ve configured the driver with the options specifying that it should run in headless mode, you should be able to follow any general Ruby Selenium tutorial or just read the documentation directly.
Hopefully you’ve found this helpful if you were looking for information on how to run Selenium WebDriver with headless Chrome using the Ruby programming language. If you’re looking for top notch consultants with expertise in browser automation, web scraping, and Ruby, then please don’t hesitate to get in touch!
If you enjoyed this article, then you might also enjoy these related ones.
A guide to setting up a practical proxy API on Amazon's Lambda using Node.js and Express.
An updated example of techniques to avoid detection.