IoT ETL solution using AWS Managed Services Showcase

-

Our Canadian customer involved us in designing and delivering a solution based on AWS, which aims to repeatedly collect, transform & transfer data from an IoT platform collector to the Salesforce CRM using the ETL pipeline. The ETL pipeline had to include processing and validation ingested data to insure conformity and handle exceptions/edge cases.

After gathering more specific requirements, evaluating limitations, we started preparing the architecture design proposal. The process included AWS components evaluation, Spikes, and PoCs, and finally brought as to the design, which was as per above attached scheme.

 

We proposed a Serverless solution based on DynamoDB, Lambdas, SQS, SNS, CloudWatch. After the acceptance of the proposed concept, we have developed MVP and delivered it to the AWS Infrastructure. Working MVP was successfully tested on the AWS environment against defined use cases and workloads.