Senior Data Engineer - Spark

Posted 24 August 2021
Salary£1 - £500 per day
LocationLondon, Greater London, England
Job type Contract
DisciplineTechnology Recruitment
Reference20140
Contact NameHannah Lancaster

Job description

Senior Data Engineer - Spark

Working for a leading digital insurance company on an initial 6 month contract (outside IR35), can be based remote with 1-2 days a week in London. Your role will be pivotal in delivering and owning high performance data applications using cutting edge technology. As a Senior Engineer within a Next Generation Data Platform team, you will be driving the development of quality solutions for customers and be proactive in adopting and championing best practices.

Essential Skills:
*Strong experience in Data Engineering - building strong, robust pipelines to ingest data from a variety of sources with a focus on real-time streaming
*Strong Spark / Spark Streaming experience
*Experience in Java, Kafka and Streaming architectures
*Experience in Kubernetes, Docker, and AWS
*Demonstrable experience for modern software engineering principles (Agile, CI/CD, TDD/BDD)
*Active learner, keeps up to date with industry knowledge and keen to learn new languages and tools
*Self-motivated, natural leader and motivator. Passionate about Technology
*Customer focused
*Great communicator, influencer and negotiator

Desirable Skills:
*Scala (we would consider a strong Scala Data Engineer who has a keen interest in developing their skills in Java)
*EMR & Apache Airflow
*Python