Lead Data Engineer

  • Company:
  • Location:
  • Salary:
    negotiable / month
  • Job type:
  • Posted:
    3 days ago
  • Category:

Prospa is driven by its people. Their diversity, skills and passion are the foundation of what we deliver to our customers – where employees are dedicated to achieving stellar results and exceeding customer expectations. Our values: obsess about customers; deliver value fast; day 1; simplicity; be bold, open and real; and one team, inform how we think and act every day. We are also recognised as an AON Hewitt Best Employer in 2017 and 2018 FINNIES Best Fintech Place to Work and a Great Place to Work in 2019.

Prospa helps small businesses across Australia achieve their dreams, by crafting an alternate funding pathway when traditional finance is not available. To date, we’ve helped over 24,000 unique small business customers across Australia with over $1.4 billion in small business loans.

Prospa has established itself as a clear #1 online lender to small business in Australia and current growth trajectories see the business continuing to invest in its rapidly growing team, currently 210 strong.

This represents a unique opportunity to join one of Australia’s most exciting and fastest-growing Fintech businesses with an awesome culture and plenty of further growth.

Prospa is extremely passionate about its people. Their diversity, skills, and passion are the foundation of what we deliver to our customers. Our people, in turn, are driven by our values.

The role

As a Staff Data Engineer, you will be part of our Data Engineering team. In your role, you will work closely with Data Analysts, Data Scientists, Product Managers and Software Engineering teams. You will be responsible for extending and optimizing our data platform and architecture to provide a reliable and trustworthy data as a service offering. Your work will ensure our data pipelines are running efficiently, our enterprise data lakes and data warehouses are up to date and operating reliably, and data is presented to our analysts, scientists, and stakeholders in an easy-to-use, trustworthy, and self-serviceable manner.

You will be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. Working with your fellow Data Engineers to solve business problems, provide training on tooling, sharing knowledge on making sense of data, and forge trusted ongoing engagement with our breadth of stakeholders. You will need to love the prospect of a fast-paced environment, working with a wide range of data tools, and optimizing and bringing our data architecture into maturity to support our growing business needs.

Key responsibilities include

  • You will extend and optimize our information models to meet BI and Data Science needs. In addition, our data pipelines, data lake, and data warehouse. Catering to our ever-growing set of data sources, with streaming and batch processing needs.
  • Building reusable tools in Python/Javascript/etc that will allow the wider team to deliver value fast with precision.
  • Driving the adoption of leading practices and suitable technologies in developing end-to-end data solutions within Prospa’s cloud environment.
  • Partnering closely with a diverse set of stakeholders to solve data and business problems with self-service and re-usability in mind.
  • Be accountable for the availability, reliability and data quality of our data platforms end-to-end.
  • Drive the quality, surfacing data and metrics to guide development, quality and availability.
  • Mentoring Prospa Data Engineers, Data Scientists and Data Analyst through peer programming, effective code reviews, and delivering feedback.
  • Engage closely with engineering, product and business partners to prioritise technical tasks alongside business objectives.
  • What you’ll need to succeed

  • Extensive experience in working with MSSQL, Databricks, Snowflakes, Airflow, Spark, Kafka and accompanying ecosystems.
  • Extensive knowledge in building enterprise data warehousing and data lakes.
  • Extensive knowledge in data modelling techniques and data workflow management.
  • Extensive knowledge of Object Oriented Programming and Test Driven Development in Python and JavaScript.
  • Experience in working with Linux-based systems, NoSQL databases and data structures.
  • Experience in working with Docker containers and Kubernetes.
  • Experience in building and maintaining CI/CD pipelines.
  • Ability to understand Prospa’s complex business practices and objectives.
  • Passion and leadership in data engineering, cloud, software development and security practices.
  • About you

  • 8+ years Data Engineering experience.
  • Have demonstrable examples of creating and maintaining:
  • Robust and extensible data pipelines for production environments.
  • Secure, performant and well-modeled data stores.
  • Reliable and efficient streaming and batch data environments.
  • Enterprise-grade Data Warehouse and Data Lakes.
  • Privacy by design across the data layers.
  • Experienced in building, scaling, and monitoring resilient distributed systems.
  • Experienced with cloud platforms, preferably Azure, in an enterprise environment.
  • Experienced working in Agile Frameworks, including Scrum and Kanban.
  • Deep experience in Python, JavaScript, and SQL.