Technologies
Alright, product itself can barely solve any problem, people using it do. Nevertheless, there are few products which we like as they provide unique value and where we developed our expertise.
Distributed data processing engine with fast execution core. It’s able to heavily optimize your transformations and run them in parallel. Suitable for small-scale data analysis, but its primary intention is to run transformations on big data. Supports multiple languages like Scala, Python, R, or custom extended version of SQL.
Spark is used by many companies including NASA. Many customers use Spark as a service provided by Databricks, Qubole or AWS.
Some of us worked for years with many technologies in DWH, ETL tools, data analytics tools, etc. But we agreed that Spark is really revolutionary in terms what it can do for a developer without requesting additional setup, especially in cooperation with cloud storage ecosystems.
Databricks is a company founded by former Spark developers. It’s also a product that provides the ability to easily provision a Spark cluster on multiple major cloud hyperscalers - Microsoft Azure, AWS, and Google Cloud Platform. A ready-to-use notebook based Spark environment is seamlessly integrated with other technologies MLFlow and Delta Lake to quickly build and operate your data lake environments, develop models and other data driven activities.
Proof of the platform's popularity is the number of customers from different industries. We like it as it can provide a cloud-agnostic, scaling calculation cluster priced based on real use with minimal maintenance effort.
Model Governance Suite is our product targeted primarily to banks, insurance agencies and organizations governing higher number of risk models, EUCs or algos which need to be regularly updated, monitored and validated by distributed teams.
Well, everybody knows the AWS. Very versatile, powerful, we use it as a platform for several internal and customer's projects.
"Old" good Java - considered by many: obsolete, talkative, slow, new COBOL. Alright, maybe, but still in many disciplines unbeatable platform of our choice.
