Senior Software Engineer

Description

Role Overview

As a Software Engineer, you will be responsible for developing software frameworks, drivers, and tools that accelerate open-source big data processing frameworks on custom hardware accelerators. Your work will focus on enabling and optimizing large-scale data processing workloads in tailor-made cloud environments, ensuring seamless integration with AWS-based data processing services. You will develop and optimize software stacks, implement benchmarking, debugging, and performance analysis tools, and evaluate acceleration gains to enhance efficiency and performance in cloud-based data platforms.

Requirements

Qualifications & Skills

Required Qualifications:

• Bachelor's degree in Computer Science, Electrical Engineering, or a related field from a leading university.
• 5+ years of experience in software development, focusing on high-performance computing.

Preferred Qualifications:

• Experience in runtime system development and low-level performance optimization. 
• Strong background in debugging complex system environments. 
• Experience implementing and optimizing multi-threaded algorithms.
• Solid understanding of columnar storage formats (e.g., Parquet) and their impact on query performance.
• Familiarity with AWS cloud services, including AWS EMR, S3, and EC2.
• Proficiency in Python or Scala for data processing and system development. 
• Familiarity with hardware accelerators.