Intoli Blog

The Red Tide and the Blue Wave: Gerrymandering as a Risk vs. Reward Strategy

An interactive explanation of how gerrymandering is a risky strategy that allows for the possibility of a blue wave.

Continue reading

Performing Efficient Broad Crawls with the AOPIC Algorithm

Learn how to estimate page importance and allocate bandwidth during a broad crawl.

Continue reading

Breaking Out of the Chrome/WebExtension Sandbox

A short guide to breaking out of the WebExtension content script sandbox.

Continue reading

User-Agents — Generating random user agents using Google Analytics and CircleCI

A free dataset and JavaScript library for generating random user agents that are always current.

Continue reading

How F5Bot Slurps All of Reddit

The creator of F5Bot explains in detail how it works, and how it’s able to scrape million of Reddit comments per day.

Continue reading

No API Is the Best API — The elegant power of Power Assert

A look at what makes power-assert our favorite JavaScript assertion library, and an interview with the project’s author.

Continue reading

Recreating Python's Slice Syntax in JavaScript Using ES6 Proxies

A gentle introduction to JavaScript proxies where we use them to recreate Python’s extended slice syntax.

Continue reading

A Slack Community for Developers to Discuss Web Scraping

Intoli is launching a new Slack community called Web Scrapers where developers can chat about web scraping.

Continue reading

Building a YouTube MP3 Downloader with Exodus, FFmpeg, and AWS Lambda

A short guide to building a practical YouTube MP3 downloader bookmarklet using Amazon Lambda.

Continue reading

Running FFmpeg on AWS Lambda for 1.9% the cost of AWS Elastic Transcoder

A guide to building a transcoder using Exodus, FFmpeg, and AWS Lambda.

Continue reading