Find Jobs
Hire Freelancers

Expert to build a web using Scrapy + djangoCMS + AngularJS4

$130-185 NZD

종료됨
게시됨 5년 이상 전

$130-185 NZD

제출할때 지불됩니다
IMPORTANT NOTES: 1) Please read the project details and attachment before you bid. 2) Full requirements as in attachment, which is fairly clear 3) Budget currency is NZD 4) Total price of this job is as high as you bid, no-negotiable. sources: 1)[login to view URL] 2)[login to view URL] ASUMPTIONS: A1: 1) scrap by using regex to complete the ID for the whole URL A2: 2) can only be scrap based on looping each category/subcategory to get all listings in each page If you have a better way for A2, that is great. --------------------------------- Functional requirements: --------------------------------- R1. scrap all categories (see Appendix 2: product categories in attachment) R2. scrap all listings of a website R3. scrap production information and delta information of an existing listing: price, stock^, sold qty^, see Appendix 1:schema tables below. R4. Admin backend UI to: 4.1 list all existing sites, last ran schedule and add new site(name,site url) 4.1.1 a button for pop up to set up schedule to start scrapping a site and a button to start scrap the site 4.1.2 a button to drill down the siate and redirect a new pate for displaying categories of a site 4.2 add new category in 4.1.2 and a button to scrap new category 4.3 display all listing under the category. 4.5 a page to display fields and setting for a scrapy item. Modify existing css rule for existing field of a listing 4.6 add a new css rule to scrap a new field(document/write HOWTO steps for changing the code if needed) ^: if field/data exists in page R5. webAPI to return result of listing query R6. front end pages to: 6.1 display all categories of a site on left and display all listings a site - paged 6.2 click on one listing in 6.1 and open a new page to display detail of a listing. product name, brand, image, latest price/price history(scrapped by #R3) R7. user login and add listing to watch list --------------------------------- Technical requirements: --------------------------------- Backbones: Scrapy + djangoCMS + MySQL + SQLAlchemy + AngularJS4 A. add scrapy module as an app for djangoCMS 1. support multi-threading 2. anti-scraping: [login to view URL] tool:RandomProxy + RotateUserAgentMiddleware 3. each listing has an Unique identifier(foreign key for other tables) 4. data access layer: SQLAlchemy B. admin backend 1. support login( djangoCMS support by default) 2. four *admin* pages integrated to *djangoCMS admin backend*, refer to in attachments(site page / rule page/ product page/ users) C. WebAPI as an app for django (for R5) 1. input: text(product name) or unique identifier 2. return: array of listing in JSON format [{Id, productName, brand, last price, last date, image url, [category 1, category 2, ...]}] 3. authentication token(64bit) 4. security: SSL/https D. front end to display (for R6) 1. UI(Angular4 + Bootstrap) 2. security: SSL/https 3. call webapi to retrieve data 4. support CORS: [login to view URL] 5. listing page to display products, display top 20 products by default order by last scrapped time. 6. detail page to display product details E. naming convention: CamelCase or underscore F(optional). it is great if WebAPI result can be used for both admin backend and frontend, if you want, you can use WebAPI for admin backend   Appendix Appendix 1: schema tables: 1. site (id, name, baseUrl, enable, schedule, lastScrapDateTime, lastStatus) store site as source, enable flag indicate the schedule is active for scrapping, update lastScrapDateTime each time the spider ran) 2. siteRule(id, siteId, ruleName, enable, type^, url, minId**, maxId**, linkRegex^^) ^type: depend on scrap by Id or category links (see [sources] at the beginning above) .... please find more in attachment
프로젝트 ID: 17441593

프로젝트 정보

6 제안서
원격근무 프로젝트
활동 중 6년 전

돈을 좀 벌 생각이십니까?

프리랜서 입찰의 이점

예산 및 기간 설정
작업 결과에 대한 급여 수급
제안의 개요를 자세히 쓰세요
무료로 프로젝트에 신청하고 입찰할 수 있습니다
6 이 프로젝트에 프리랜서들의 평균 입찰은 $572 NZD입니다.
사용자 아바타
Hello , How are you? I just saw your project description carefully. I am very interested in your project. I have rich experience in Scrapy + djangoCMS + AngularJS4. I am ready to start your project immediately. I am sure I can satisfy you surely and complete in a short time. Please send your message so that we can discuss about your project. ************************************************************************
$175 NZD 10일에
4.9 (46 건의 리뷰)
7.0
7.0
사용자 아바타
Hi, Sir. I am a senior web developer with 5 years of experience in building website. I am very experienced in web development with PHP(CodeIgniter, Laravel, WordPress, ...), Java(JSP, Spring, Hibernate, Struts, JSF...), JavaScript(Angular, ReactJS, ...) and so on. Here are my generic working style. - Communication is first, my timezone is as flexible as you want. - Best user experience and keep pixel perfect design. - Always not satisfied with my results and try harder to improve. - Quality of code is guaranteed. (Robust and bug-free, adopt design patterns) - Bug & issue track using Github, Bitbucket and so on. I am sure I can deliver high quality product and I want to have an interview with you. Please consider working with me. I will do my best. Thanks.
$500 NZD 10일에
5.0 (6 건의 리뷰)
5.3
5.3
사용자 아바타
Hello, Hope you are doing well. The last 8 year's I have focused on high-quality Web, IOS, Android and Computer Vision development. Please see my range of skill sets below. After coming on chat discussions I will show previous work. My technical skills: 1. Web Development - Angular 1,2,4,5, React.js, , HTML/CSS, JavaScript/jQuery, Bootstrap... - Node.js, Express, PHP, Laravel, Codeigniter, Yii, CakePHP, Symfony, Zend, Django... - Twilio, Facebook, Instagram, Twitter, Google Map, Youtube, Paypal, Stripe... - MySQL, MongoDB, Redis... 2. Mobile/CV Development - Android, iOS(Swift, Object C), Cordova, Ionic Framework, React Native ... - Java, C++, C#, Python, Matlab, Machine Learning, OCR, Image Processing.. I take pride in my attention to detail, and technical expertise required to deliver high quality, bug-free code. I approach all projects with a positive attitude, and perspective that no project is too difficult and the timeframe is unattainable. I would be really appreciated, If you will open a line of chat to discuss further Thank you for your time and consideration
$144 NZD 10일에
5.0 (4 건의 리뷰)
1.5
1.5

고객에 대한 정보

국기 (NEW ZEALAND)
Wellington, New Zealand
0.0
0
4월 15, 2015부터 회원입니다

고객 확인

감사합니다! 무료 크레딧을 신청할 수 있는 링크를 이메일로 보내드렸습니다.
이메일을 보내는 동안 문제가 발생했습니다. 다시 시도해 주세요.
등록 사용자 전체 등록 건수(일자리)
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
미리 보기 화면을 준비 중...
위치 정보 관련 접근권이 허용되었습니다.
고객님의 로그인 세션이 만료되어, 자동으로 로그아웃 처리가 되었습니다. 다시 로그인하여 주십시오.