About the role

As a Senior Software Engineer for this role, you will collaborate with the founding team to contribute to the broader integration of our product with Apache Spark and maintain the solution up to date and compatible with a variety of supported runtimes.

Your contributions to our core solution will directly impact data infrastructure processing 10s of petabytes every day where Xonai is being deployed.

What you will do

• Implement shims and backport changes required to maintain compatibility with multiple Spark versions.
• Implement or extend features at the driver level such as performance reports, usage metering, integrations with third party libraries and application configuration mechanisms.
• Improve and extend various internal tools used for benchmarking and assessing application performance.
• Independently and diligently write, test and deploy production code driven by modern software engineering practices.
• Work with the internals of leading open-source software for Big Data analytics as part of our core product for the cloud.

What you will bring

• 3+ years of experience working with large applications written in Scala.
• Strong Scala and Java programming skills.
• Experience in navigating through the abstraction layers of large codebases to get down to the relevant details.
• Strong grasp of computer science fundamentals such as algorithms and data structures.
• Critical thinking and ability to work independently without direct supervision.

Nice-to-haves

• Entrepreneurial spirit and previous experience in early stage startups.
• Experience with CI and CD tools such as GitHub actions.
• Understanding of Big Data file formats (e.g. Parquet, ORC, Avro).
• Understanding of Big Data processing engine internals.