We are looking for someone who can create a PHP Spider that can:
- Spider a website based upon just giving it a specific URL and the spider will follow all internal links it finds OR via an XML file of specific pages to spider
- It will only index the pages if a specific filetype is on them (i.e. an .flv). It will check the filesize of the file and make sure it over a certain size before indexing, also check the height and width of the object to make sure its at least a certain dimension. It will then get the unique identifier of the file on the webpage.
- When indexing it will grab the meta tags (title, keywords, description), as well as the first x amount of characters displayed on the page.
- All of this will be entered into a MySQL database.