Hạn nộp: 11/03/2025
Thời gian đăng: 08:42 12/02/2025
Lượt xem: 30
Nơi làm việc: Hồ Chí Minh
Chức vụ: Nhân viên
Hình thức làm việc: Toàn thời gian cố định
Số lượng cần tuyển: 1
Yêu cầu độ tuổi: 22 - 29
Ngành nghề:
Công nghệ thông tin1. Professional Scraping System Development
Technical Requirements:
System Architecture:
Design cross-platform Python crawling scripts
Build scalable systems
Develop parallel crawling solutions
Manage large, multi-threaded data streams
Technologies:
Scrapy, BeautifulSoup
Selenium
Asyncio, Multiprocessing
Proxy management
IP rotation techniques
2. Data Processing and Normalization
Processing Methods:
Develop API data cleaning processes
Data transformation algorithms
Integrity checks
Remove noisy data
Tools:
Pandas
Data validation techniques
Machine Learning preprocessing
3. Database Management
Specialized Skills:
Advanced SQL:
Complex queries
Performance optimization
4. Monitoring & Optimization
Strategy:
Manage scraping system operations.
Track scraping performance
Challenge handling:
IP blocking
Speed limiting
CAPTCHA