Senior Data Engineer: Data Pipelines Infrastructure

Job Description

At The New York Times, we take data seriously and use it to make our decisions, improve the quality of our journalism, and provide the best possible experience to our readers. To ensure this, we built and continue to evolve our big data platform. As we grow our analytics and data science capabilities, and consequently, the number of users, we need to grow our platform to support new challenges, opportunities, and goals.

About the Team

The Data Pipelines Infrastructure team, as the name suggests, builds all the infrastructure around data pipelines at The Times– batch and streaming. Our mission is to provide our users–technical and non-technical– with the best toolset possible and allow them to create pipelines of any complexity. The team is still in the early stages of development, so joining us means shaping the vision and architecture of the next generation of data transport, collection, and orchestration at The Times to better help develop data-driven products and features at a pace that matches our incredible growth.

As an engineer on the team, you will:

  • Build a fault-tolerant, secure, and scalable infrastructure

  • Architect, test, release, and support new functionalities

  • Build self-service platforms to power The Times' Technology and Data teams

  • Partner with other teams to develop services using storage layer, streaming, and batch data

  • Engage with our users to better understand their pain points, observing patterns, and identify opportunities for improvements

Required Experience

  • BS in Computer Science or equivalent engineering experience

  • Complete experience with the software development lifecycle

  • Experience with Python, object-oriented design, and agile engineering practices– like test automation, continuous integration/delivery

  • Experience shipping high-quality & performing systems

  • Knowledge of distributed systems principles (consistency and availability, durability, fault-tolerance)

  • Experience with the different data infrastructures–RDBMS, Columnar, NoSQL, and file-based storage solutions.

Bonus Points

  • Experience with Golang

  • Experience with Apache Beam for data processing

  • System observability using Datadog

  • Cloud-based (Google Cloud Platform) architecture

  • Experience with the following technologies:

  • PubSub and Kafka

  • BigQuery

  • Airflow (Google Cloud Composer)

  • Cloud storage: S3, GCS, …

  • Experience with agile/lean projects

You'll want to work with us if you:

  • Love building data infrastructures

  • Take ownership and support solutions you develop

  • Are enthusiastic about technology and love to grow as an engineer

  • Enjoy mentoring more junior engineers

  • Value collaboration with other members of the team

  • Have a product mindset (outcomes over output)

  • Communicate well

The New York Times is committed to a diverse and inclusive workforce, one that reflects the varied global community we serve. Our journalism and the products we build in the service of that journalism greatly benefit from a range of perspectives, which can only come from diversity of all types, across our ranks, at all levels of the organization. Achieving true diversity and inclusion is the right thing to do. It is also the smart thing for our business. So we strongly encourage women, veterans, people with disabilities, people of color and gender nonconforming candidates to apply.

The New York Times Company is an Equal Opportunity Employer and does not discriminate on the basis of an individual's sex, age, race, color, creed, national origin, alienage, religion, marital status, pregnancy, sexual orientation or affectional preference, gender identity and expression, disability, genetic trait or predisposition, carrier status, citizenship, veteran or military status and other personal characteristics protected by law. All applications will receive consideration for employment without regard to legally protected characteristics. The New York Times Company will consider qualified applicants, including those with criminal histories, in a manner consistent with the requirements of applicable state and local "Fair Chance" laws.

Location
United States
Skills
  • Google
  • Python
  • S3
  • GCS
  • NoSQL