Easy Connect [the crawler]
Software Specification The purpose of this document is to describe the required software development in order to be created a software instrument (a tool for crawling) that includes the following parts:
• Gathering information for companies and making profiles of them with the products they offer. (Crawling)
• User interface for managing – the crawler and the information which is gathered.
• Sending offers via e-mail.
• Sorting the results.
Date: 20.09.2015
Version: 1.0
Document Owner: todor.panayotov@gmail.com
Software Architecture:
As a solution we are using three different types of crawlers written on Perl/ Python or NodeJS.
Redis as a temporary storage while we are building the companies profiles.
NoSQL and Node Webkit for the User Interface.
• The following Diagram 0-A explains how the tool should work on:
Diagram 0 A
Crawlers and back end:
This chapter describes why we need three types of crawlers and Redis as middle layer storage…
Software Specification The purpose of this document is to describe the required software development in order to be created a software instrument (a tool for crawling) that includes the following parts:
• Gathering information for companies and making profiles of them with the products they offer. (Crawling)
• User interface for managing – the crawler and the information which is gathered.
• Sending offers via e-mail.
• Sorting the results.
Date: 20.09.2015
Version: 1.0
Document Owner: todor.panayotov@gmail.com
Software Architecture:
As a solution we are using three different types of crawlers written on Perl/ Python or NodeJS.
Redis as a temporary storage while we are building the companies profiles.
NoSQL and Node Webkit for the User Interface.
• The following Diagram 0-A explains how the tool should work on:
Diagram 0 A
Crawlers and back end:
This chapter describes why we need three types of crawlers and Redis as middle layer storage…