1) Need to scrap google search data for the keywords (80000) using a high performance multi-threaded asynchronous scrapers (300 to 500 threads) using proxy server that can handle IP rotation (need your suggestion which server we can use).
Keywords are stored in a table, Each thread will pick 1 keyword and mark status as processed (1) and insert the start time and scrap 50 (configurable) sites from google position 20 to 70 (from –to position are configurable) and scrap following data
Title , Description , URl , Domain Name
Insert all data in Master table and also when all 50 sites are scraped update finish time in Keyword table and threads pick up next keyword available where status is not equal to 1
Data fields of Keyword Table (MySql)
Domain Name , Subdomain name , Keywords , Category 1, Category 2, Category 3, Category 4, Sites scraped , Status , Start time , Finish time.
Data Field of Master Data (MySql)
Domain Name , Subdomain name , Keywords , Category 1, Category 2, Category 3, Category 4, Scraped Title , Scraped Description , Scraped Url, Scraped Domain name, Account Id, User ID ,User Name , Expiry Date.
Domain Name , Subdomain name , Keywords , Category 1, Category 2, Category 3, Category 4 you will get from Keyword table.
Scraped Title , Scraped Description , Scraped Url , Scraped Domain name you will get using your script
Update account Id as 1, User Id as 9810822422, User Name as TTPL Update Expiry Date as 01-01-2050 for all keywords.
This is a one time activity to fill Master table for our research.
2) Need just a function (PHP) to get 50 urls (Google) for a keyword starting from position 20 in an array . (no scraping , nothing).
3) Need just a function (PHP) to get the google ranking for a keyword for a website.
Hi! Hope you are doing well.
I read your project description carefully. I am professionalPHP programmer with skills including Data Mining, Web Scraping, Python, Data Scrapingand PHP.
I already do many project in PHP.I have 10+ years experince in development.
Please send a message to discuss more about this project.
Thank you
Dear Employer, My name is Aishwarya and I’m a Virtual Assistant. I have Four Years of experience in Lead Generation, Email Marketing Data Scrapping, and LinkedIn Research and Lead generation. I have completed many higher Research projects from the USA, Australia, and Europe Countries. I would love to help you out! Thanks, Aishwarya Kapile.
Hello.., My availability: 40+ hours/week. Ready to start work immediately. I read your project post of fullstack Python Developer to Scrap google search data for the Keyword
I am fullstack Python developer having skillsets in Python Django/Flask, Machine Learning, Data Science.
I have a strength to offer 24/7 support including weekend and night time and excellent software quality work and provide the best support on your project.
Please message me to start discussion.
Thanks
"PubMed"
Hi ,
I am a web scraping specialist.
I have rich experience about web scraping.
I've been using bs4, selenium, Scrappy and LXML etc.
I've scrapped dozens of sites at once also.
We can chat and discuss in detail about project.
Thanks for consideration.