Kindgeek

Datasembly

Datasembly provides access to millions of grocery and retail pricing records aggregated from hundreds of grocery delivery and online shopping websites. This data is hyper-local, highly volatile, and notoriously difficult to collect, but using specialized technology Datasembly aggregates and transforms this data so organizations can use it effectively. These prices currently power economic indicators, market intelligence, and consumer apps.

Platform:

Web

Technologies:

Scala, Akka, Slick,

Tools:

Redshift, Play libs

Country of Origin:

USA

Year completed:

2016

Project Duration:

2 years

Industry:

Trade

Size of Team:

Development approach

To achieve our goals we implemented such things:

-The data is stored in AWS Redshift
-We use Postgres for real-time aggregations during scraping
-Play libraries are used for making http requests
-Play JSON and JSoup libraries for Json and HTML parsing
-AWS Kinesis for moving the data into Redshift
-Spark for populating Cassandra db

Results achieved

UNPRECEDENTED ACCESS TO GROCERY AND RETAIL PRICES
collected from the web
Prices are collected for each store and zip code, allowing you to
track who is selling what, where, and for how much
Datasembly's up-to-date data empowers you to make fast, informed,
data-driven decisions. Don't let stale data ruin your analysis
Prices, availability, and promotion data, covering all your
geographies, all your products, and all your competition