Scrapy Jobs

I have a list of subreddits and need a one-off extraction of three specific data layers from each of them: • post titles with the body text • every comment thread • publicly available user info tied to those posts and comments (username, karma, join date where the API allows) •any details relative to any contact information for individuals that work in SBA EIDL service department Please return everything in a clean, well-structured CSV—one file per subreddit or a single consolidated file with clear subreddit, post, and comment identifiers. A reusable script is important to me. Python with PRAW, Pushshift, Scrapy, or any Reddit-API friendly tooling is fine as long as you document the install steps and rate-limit handling so I can rerun it later without...

I need a reliable web-scraping solution that pulls up-to-date market statistics focused on individual wrestlers. The target sites are public sports portals and federations that list match results, rankings, and season performance; I’ll supply the URLs as soon as we start. The scraper must extract for every athlete: full name, weight class, team or club, most recent match outcome, cumulative win-loss record, points scored, and ranking movement over time. I want the data normalised into a single CSV and a companion JSON feed so it can drop straight into my analytics pipeline. Python is my usual stack, so Scrapy, BeautifulSoup, or a light Selenium layer for the occasional dynamic page all work. Please build in polite rate limiting, user-agent rotation, and a quick retry strategy so th...

I’m running several CPA campaigns and I need a fresh batch of email contacts that are guaranteed to pass Debounce (or an equivalent) with a very low bounce rate. The focus is strictly on email leads sourced from two places: reputable business directories and well-known e-commerce sites. Here’s what success looks like for me: • You pull the data ethically and accurately, respecting each site’s terms. • Every address is verified through Debounce (or a comparable validator) before delivery. • Final output arrives in a clean CSV or Excel file with, at minimum, email, source URL, and any publicly available name or company field you can capture. If you already have an efficient scraping workflow—Python + Scrapy/Selenium, Octoparse, Apify, etc.&m...

I need an automated scraper that gathers data from several news sites in near real-time. The tool should loop through a list of URLs I will provide, respect each site’s where possible, and export the captured information to CSV or JSON so I can feed it straight into my analysis pipeline. I’ll share the exact fields during kickoff, but the scraper must be flexible enough to handle common article elements—headline, body text, author byline, publication date, and source URL—and easy to extend if I add more outlets later. Time is critical. Delivery within 24–48 hours is preferred, so please lean on a proven stack such as Python with Scrapy/BeautifulSoup, Node with Cheerio, or any robust alternative you already master. The script should: • Rotate user agent...

I already have a Python-based Scrapy project and now need dedicated spiders that can harvest data from the mobile apps Rabbitmart (Egypt), Voo (Egypt) and Oscar (Egypt). The crawlers should pull every publicly available endpoint (or reverse-engineered one) required to deliver: product details, user reviews, pricing information, the underlying product IDs, any active promotions or discounts and—where the apps expose it—current stock levels. All captured information must be written to clean, well-structured JSON files so that I can feed it straight into my existing pipeline. I’m still unsure whether the apps will demand login or token-based authentication; therefore, please build the spider with the flexibility to plug in credentials, headers or session cookies if we disc...