Challenge : The client wanted continuous updates on its products’ prices, and reviews on them as available on its customer e-commerce websites. There were about 20 of such popular customer sites to track daily for the client to report internally and perform analyses on trends in pricing by customer as well as to understand how their products were doing in the market.
Solution : Around 5000 client products were scraped daily for pricing, after that reviewed and then delivered to the client in multiple CSV’s. Each record contained the product details as on customer site, product details as on the manufacturer’s site and pricing/review details. Since the data volumes were small, the client could directly import this data into excel and perform their analyses.