Web Scraping Engine - 5 Sites Initially

We are launching a new aggregation service for specialty financial data. The service will work just like Yodlee but serve different data related to financial transactions. All sites are secured with credentials, which we will store, and some have challenge questions. We need a web scraping engine developed that can operate in real time and in batch. The engine must be able to scale to an enterprise level and be multi-threaded. We will store the user credentials so the engine must be able to cycle through different credentials to gather the data for that user and populate a database. Some of the sites are not straight-forward and use Ajax and other technologies that make scraping more difficult. For this project we will want five sites scraped initially but will add many more if the engine works properly. In short, we are looking for web scraping experts that can develop not only the initial product but also help maintain it and add sites. We can provide some initial site credentials as samples if you like. We have developed a proof of concept using the technologies listed below but we are open to whatever works. Selenium Webdriver PhantomJSDriver C# Webscraping MSSQL Entity Framework Asp.NET MVC Asp.NET Web API HTML Agility Pack ITextSharp