Evaluating bids

Data Mining/Web Scraping

Published on the July 23, 2024 in IT & Programming

About this project

Open

For this project I am looking for someone to scrape 500k+ web pages (all in one location - a list) for simple data extraction.  This is all public information with simple formatting.  As an example, it is a list of 50 lines and about 4000 pages, each line with a profile page.  You will need a crawler to open every profile on every line (5) for all 4000 pages.  Then each page will have to be scraped for basic data.  I need this example done about 50 times over.  Yes, a lot of data but if you know what you are doing, this can be completely automated. This can lead to future opportunities as well.

Category IT & Programming
Subcategory Web development
What is the scope of the project? Small change or bug
Is this a project or a position? Project
Required availability As needed
API Integrations Other (Other APIs), Cloud Storage (Dropbox, Google Drive, etc.)
Roles needed Other
Other roles needed Build the crawler and scraper and/or use an existing one and extract the data. If I had the time I would do this myself it's not a very complicated gig.

Delivery term: August 16, 2024

Skills needed

JavaScript MySQL PHP

Want to start working on this project?

Log into your Workana account or Register today. Boost your freelance work!

Other projects posted by A. H.