Expert in Web Crawler Development

**Job Description: Expert in Web Crawler Development**

**Position:** Web Crawler Developer (Freelancer)
**Work Model:** Remote
**Compensation:** Negotiable (based on milestones + fixed price per domain)

#### **About the Role:**

I am looking for an experienced developer specializing in **web scraping and crawler development**. The crawler must be capable of extracting structured data from various websites, with each domain requiring individual customization.

If you have experience and can develop professional, scalable solutions, I look forward to your application!

#### **Requirements:**

? Extensive experience in **web crawler development** and **scraping technologies**
? Proficiency in **Python (Scrapy, Selenium, BeautifulSoup)** or similar technologies
? Experience with **IP rotation, CAPTCHA handling, and proxy usage** to avoid bans
? Ability to export **structured data (JSON, CSV, SQL, APIs)**
? Optional experience in **developing a dashboard** for managing scraped data

#### **Responsibilities:**

???? Development of a powerful **web crawler** for various domains
???? Customization to specific website structures, including **CAPTCHA handling & anti-bot technologies**
???? If possible: Development of a **dashboard with extensions** for managing and visualizing collected data
???? Providing a **pricing overview for individual milestones**
???? Fixed price offer for new domains (ranging from simple to complex websites)

#### **Compensation & Collaboration:**

???? **Payment based on milestones** + **fixed price per new domain**
???? Potential for a long-term collaboration as new domains will be added regularly

If you are interested, please send me a message with the following details:

- Your **experience & reference projects**
- The technologies you use for scraping
- Whether you can develop only the crawler or also a dashboard
- Your **pricing expectations** for individual milestones

I look forward to your response!

**Best regards,**
David