Apply Infrastructure Engineering

Senior Big Data Infrastructure Engineer

Location : Dallas, TX

Do you want to tackle the biggest questions in finance with near infinite compute power at your fingertips?

G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to nurture their talent in a dynamic, flexible and highly stimulating culture where world-beating ideas are cultivated and rewarded.

This is a hybrid role based in our new Dallas infrastructure hub where we work on the latest technologies in a cutting-edge environment.

The role

G-Research is seeking a DevOps focused Infrastructure Engineer, with deep experience in HDFS, YARN and the Hadoop ecosystem, to join its growing Big Data Platform Engineering team.

Big Data Platform Engineering manages a significant distributed Cloudera HDFS storage footprint, has recently adopted Airflow and has also started to explore other Open Source offerings, such as Ozone. In addition, there is also an overlap in technology support within the team, in products such as Trino, Spark, Grafana, Hive, Nifi and HBase to name a few.  

As a Senior Big Data Infrastructure Engineer, you will help us deliver automation, feature enhancements and best practice security on our existing hybrid cloud platforms, as well as explore new technologies and integration with other Open Source products within our ecosystem.

There is a significant focus on configuration management, container orchestration, Infrastructure as Code (IaC) and CI/CD. In return, you will gain exposure to the latest hardware and software technologies in a forward-thinking company, which values innovation, personal development and training.

Key responsibilities of the role include:

  • Developing, deploying and maintaining big data solutions within the Hadoop ecosystem, such as HDFS, YARN, Ranger and Hive.
  • Delivering engineering solutions ensuring they are scalable, stable, performant, and operationally efficient
  • Providing metrics, documentation, and self-service infrastructure to help our customers work at pace and get the most out of the platforms
  • Proactively identifying and addressing system performance bottlenecks, security vulnerabilities, and areas for improvement to ensure systems are stable and robust
  • Deploying and maintaining Airflow
  • Participating in the out of hours rota, providing support and maintenance for our critical systems across the BigData ecosystem

Who are we looking for?

Whatever your technical background, good coding ability is essential for this role.

The ideal candidate will have the following skills and experience:

  • Strong HDFS, YARN and Hadoop ecosystem knowledge, preferably on the Cloudera Distribution but also OSS Distributions
  • Excellent coding skills, preferably Python
  • Experience working with CI\CD pipelines, such as Jenkins and ArgoCD
  • Experience working with Configuration Management tools, such as Ansible and Terraform
  • Strong Linux, Hardware and Networking knowledge
  • Experience deploying and running applications on Docker and Kubernetes, including the creation of Helm charts
  • Experience working with distributed systems

Highly desirable but not essential:

  • Experience working with Ozone
  • Experience working with other Big Data technologies, such as Spark, Trino, Hive, Airflow, Ranger, Oozie etc
  • Data Engineering experience (Spark application development, SQL)
  • Team Lead experience 

Why should you apply?

  • Market-leading compensation plus annual discretionary bonus

  • Informal dress code and excellent work/life balance

  • Excellent paid time off allowance of 25 days

  • Sick days, military leave, and family and medical leave

  • Generous 401(k) plan

  • 16-weeks' fully paid parental leave

  • Medical and Prescription, Dental, and Vision insurance

  • Life and Accidental Death & Dismemberment (AD&D) insurance

  • Employee Assistance and Wellness programs

  • Generous relocation allowance and support

  • Great selection of office snacks, and hot and cold drinks 

  • On-site gym and car parking

G-Research is committed to cultivating and preserving an inclusive work environment. We are an ideas-driven business and we place great value on diversity of experience and opinions.

We want to ensure that applicants receive a recruitment experience that enables them to perform at their best. If you have a disability or special need that requires accommodation please let us know in the relevant section.

Apply

Stay up to-date with G-Research

Subscribe to our newsletter to receive news & updates

You can click here to read our privacy policy. You can unsubscribe at anytime.