Python Developer - Web Scrapin · Noida

Manupatra is seeking a skilled Python Developer in Noida to build robust web scrapers, crawlers, and automated data extraction pipelines for large-scale legal data processing.

Key Responsibilities

• Design, build, and maintain scalable web scrapers and crawlers to extract data across diverse sources.
• Parse, clean, and structure unstructured web content into standardized database formats.
• Implement multithreading and multiprocessing architectures to maximize crawler throughput and efficiency.
• Handle complex scraping challenges including dynamic JavaScript rendering, rate limits, and CAPTCHAs.
• Manage and store extracted data efficiently using MySQL and MongoDB databases.
• Integrate diverse external datasets into centralized database repositories.
• Write monitoring and alerting scripts to oversee crawler health and analyze system logs.
• Troubleshoot, maintain, and optimize legacy scraping scripts when source website layouts change.
• Ensure the security, reliability, and continuous performance of all extraction workflows.
• Follow core software design principles to create clean, reusable, and modular Python code.
• Document web crawling logic, extraction schemas, and pipeline architectures.