Alteryx is more than just a data prep tool. It can also be used for webscraping. This week, I’m going to show different methods of webscraping using Alteryx, ranging from easy to slightly more difficult. Part one will show how to webscrape using Alteryx only, I use webscraping Wikipedia as an example. Part two will show webscraping using the python tool in Alteryx and we will webscrape and parse information from Google search result.
In this post, I will walk you through how to scrape this Wikipedia page. There are multiple ways to webscrape in Alteryx such as using Text to Column & Formula tool, using REGEX (regular expression), or using an unofficial tool called HTML/XML Parser. Text-to-Column + Formula tool is the simplest method, you can refer to this Alteryx Challenge’s solution for the details. REGEX is quite difficult and requires a bit of practice to get the hang of it. As a result, in this post, I will focus on the last and also the quickest method. HTML/XML Parser is developed by my colleague, Laszlo (refer to his blog post on the process of how he created it). You can download it here.
Step 1: Download HTML page
To download the HTML page, first, use the text input tool to paste the URL. Then, we use the download tool to download information from the url. Alteryx will not display the full result so you’d have to use the Browse tool to see the full downloaded data. As I mentioned above, the download tool in Alteryx doesn’t work for all webpages so you should check whether the information you want is in the downloaded data. We can do that simply by copying everything into Notepad and press Ctrl+F to find a keyword.
Workflow
Output from Browse
Step 2: Inspect the page
After we download the data, we now need to parse or extract the information. To do that, we need to understand the structure of the data by inspecting its elements. Right-click on the table, and select inspect.
Step 3: HTML Parser to split into rows
Step 3: HTML Parser to split into columns