About the Role
As a Data Engineer on the VMA project, you will work with multiple teams to deliver solutions on the Azure Cloud using core cloud data warehouse tools (Azure Data Factory, Azure Databricks, Azure SQL DW and other Data related technologies). This is a Data Engineer from the application side.
Suitable candidates must be able to analyse data and develop strategies for populating data lakes. This is not an infrastructure position. This person may be called upon to do complex coding using SQL, R, Python and SAS-SQL.
Work as part of a team to develop Cloud Data and Analytics solutions;
Lead the development of cloud data warehouses, data as a service, business intelligence solutions;
Data wrangling of heterogeneous data;
Ability to provide solutions that are forward-thinking in data and analytics;
Deliver a quality product;
Coding complex U-SQL, Spark (Scala or Python). T-SQL; and;
Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Data Factory, Azure Databricks).
We are seeking candidates with demonstrated banking experience and familiarity with regulatory, risk and compliance areas.
Additional skills required include:
Expertise in any ETL tool i.e. (SSIS, Informatica, Data Stage);
Expertise to Implementing Data Warehousing Solutions;
Experience as Data Engineer in Azure Big Data Environment;
Programming experience in Scala or Python, SQL;
Hands-on experience in Azure stack (Azure Data Lake, Azure Data Factory, Azure Databricks) — Mandatory;
Excellent understanding of other Azure services like Azure Data Lake Analytics & U-SQL, Azure SQL DW;
Excellent understanding of Azure Databricks platform and can build data analytics solutions to support the required performance and scale;
Demonstrated analytical and problem-solving skills, particularly those that apply to a big data environment;
Excellent Understanding of Modern Data Warehouse/Lambda Architecture, Data warehousing concepts;
Proficient in a source code control system such as GIT;
Knowledge of C# and ability to code; and
Ability to use word to create required technical documentation.