Project name: Data Extraction
There are approximately 37,000 childminders in the United Kingdom, their contact details are publicly available on Local Authority Council directories. The client needed details of childminders from 132 of the 206 available Local Authority Council directories. The details needed to include contact (childminder’s name, name of childminder setting, email address, telephone and mobile numbers); childminder’s OFSTED number; whether they delivered funded places for 2-year-olds and 3-4-year-olds; whether they delivered 30 hours’ free childcare; the local authority name and the childminder’s profile link.
In order to provide actionable results, we used data extraction techniques and tools.
The data extraction process
Our data analyst was provided with 132 websites from which we needed to record approximately 18000 childminders’ details. The first step was navigating the homepage of a website and working out on how to locate childminders only out of the different service providers provided. Next was to figure out how to obtain the contact details of each of those childminders.
Albeit the task could have been accomplished by simple copy and pasting, it would have been heavily time-consuming, highly inefficient and would have made it impossible to deliver project results within the set deadline.
For increased efficiency, we resorted to using an automated web data extraction tool, in this case, the chrome web scraper, which runs in the Google Chrome browser. Using this tool, we scraped multiple websites at the same time and successfully collected all the childminders’ details needed. Once the data was scraped, we downloaded it as a CSV file that I further imported into Excel and did final editing and formatting.
The chrome web scraper offered us the ability to completely automate web data extraction. We scraped the provided websites successfully while remaining within the project deadline. In the end, details of 18266 childminders in the United Kingdom were gathered and presented to the client, who was more than satisfied with the outcome.