+ Post Job +
Home Data Engineering & BI

Senior Big Data Engineer

📍 Ahmedabad 🏷️ Data Engineering & BI 💰 ₹184,000 / month

Thaltej on SG Highway in Ahmedabad has a Senior Big Data Engineer opening, full time, on site, with a data team processing volumes large enough that a naive query design shows its problems within minutes rather than hours. Six and a half years of experience working with large, real-world datasets is the baseline expectation.

The role covers transforming raw data into structured reports at scale, building dashboards for business stakeholders, maintaining reporting workflows, and optimizing queries that would otherwise take hours to run against the volume of data this platform handles daily. A senior engineer here needs real intuition for when a query is going to be slow before running it, not just after watching it time out.

What you'll be doing

  • Build and maintain big data pipelines using Hadoop and Apache Spark
  • Transform raw data into structured, query-optimized formats
  • Build dashboards and reports for business stakeholders
  • Optimize slow-running queries and pipeline jobs across the platform

Required skills are Hadoop, Apache Spark, SQL, either Python or Scala, and real experience designing data pipelines that hold up under production load rather than only working cleanly in a development environment. A bachelor's degree in computer science, information technology, or a related field is expected, along with proven proficiency in SQL, data pipelines, and BI or reporting tools.

Good to have

  • Experience with cloud-based big data platforms such as Databricks or EMR
  • Familiarity with data lake architecture and partitioning strategies
  • Any background tuning Spark jobs for cost efficiency, not just speed

Thaltej has become a dense pocket of Ahmedabad's IT and business services scene along SG Highway, and the data platform this team maintains supports multiple downstream products, which means a pipeline change here has ripple effects that need real consideration before deployment, not just a quick fix pushed straight to production.

Naukri Mitra has listed a handful of data engineering roles from this employer, and past hires describe the scale of the data here as genuinely different from smaller companies they'd worked at previously; techniques that worked fine at a smaller scale sometimes fall apart entirely once applied to datasets this size, and part of ramping up is unlearning a few habits that don't transfer.

Interviews include a technical round on Spark and pipeline design, plus a discussion of a past project where you optimized a slow-running job. Candidates who can quantify the improvement they made, not just describe the change qualitatively, tend to interview well.

Career progression from this seat tends to follow a fairly clear path: strong performers in a Senior Big Data Engineer role typically move up one level within eighteen months to two years, taking on broader ownership of Hadoop work rather than a fixed set of tickets. The team has said this progression isn't automatic and depends on genuinely stepping up during that window, not just accumulating tenure.

A Spark job that runs fine in a development environment can behave very differently at true production scale, since partition skew and memory pressure rarely show up until real data volume hits the pipeline, and senior engineers here are expected to think about scale from the start rather than optimizing only after something breaks in production.

Pipeline runs here process data on both a real-time streaming basis and a nightly batch basis depending on the use case, and senior engineers are expected to make deliberate architectural calls about which approach fits a given requirement rather than defaulting to whichever pattern is more familiar.

The data engineering team holds a monthly session reviewing pipeline failures from the previous weeks, looking specifically for patterns across seemingly unrelated jobs, since a surprising number of production issues here have traced back to the same underlying upstream data quality problem showing up in different pipelines.

Benefits

  • Health insurance covering the employee and immediate family
  • Paid time off
  • Performance-linked bonuses
  • On-site cafeteria facilities
  • Cab or commute support for the Thaltej office

This role is based at Thaltej, SG Highway, Ahmedabad, pin code 380059, five days a week on site. Pay runs up to ₹1,84,000 a month. The team hopes to fill this seat within six weeks.

Frequently Asked Questions

Designing pipelines that hold up at true production scale, since partition skew and memory pressure rarely show up until real data volume hits the system, not in a smaller development environment.
Both, depending on the use case, and senior engineers are expected to make deliberate calls about which approach fits a given requirement rather than defaulting to a familiar pattern.
Hadoop, Apache Spark, SQL, and either Python or Scala, along with real experience designing pipelines that work under production load.
A technical round on Spark and pipeline design, plus a discussion of a past optimization project where quantifying the improvement you achieved matters.
Pay runs up to ₹1,84,000 a month, and the team hopes to fill the seat within six weeks.
Apply Now