Data Engineer
Job Description
Job Description
Looking for Senior Data Engineers who are experienced, self-driven, analytical, and strategic. In this role, you will work with clients across large and complex data lake/warehouse environments. You are passionate about working with disparate datasets in order to bring data together to answer business questions. You should have deep expertise in the creation and management of datasets and the proven ability to translate the data into meaningful insights through collaboration with product managers, data engineers, business intelligence developers, operations managers and client leaders. In this role, your skills and expertise will be integral in strategic decision-making, answering complex questions by taking ownership of the end-to-end development of data engineering solutions.
Responsibilities
- Interface with project managers, business customers, data architects and data modelers to understand requirements and implement solutions.
- Design, develop, and operate highly-scalable, high-performance, low-cost, and accurate data pipelines in distributed data processing platforms providing ad hoc access to large datasets and computing power.
- Explore and learn the latest big data technologies. Evaluate and make decisions around the use of new or existing software products to design data architectures.
- Recognize and adopt best practices in data processing, reporting, and analysis: data integrity, test design, analysis, validation, and documentation.
Qualifications
- Bachelor's degree in Computer Science or related technical field, or equivalent work experience.
- 5+ years of work experience with ETL, Data Modeling, and Data Architecture.
- 5+ years of experience with SQL and large data sets, data modeling, ETL development, and data warehousing, or similar skills.
- 2+ years experience with the AWS or MS Azure technology stacks. AWS should include Redshift, RDS, S3, EMR or similar solutions built around Hive/Spark etc. MS Azure should include ADF, Azure BLOB storage, Azure Synapse etc.
- Excellent in ETL optimization, designing, coding, and tuning big data processes using Apache Spark or similar technologies.
- Experience operating very large data warehouses or data lakes.
- Experience with building data pipelines and applications to stream and process datasets at low latencies.
- Demonstrate efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.
- Knowledge of distributed systems and data architecture - design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning, and MPP of high-level data structures.
- Experience with full software development life cycle, including coding standards, code reviews, source control management, build processes, and testing.
- Industry
- Information Technology & Services
How to Apply
Ready to start your career as a Data Engineer at ThoughtsWin?
- Click the "Apply Now" button below.
- Review the safety warning in the modal.
- You will be redirected to the employer's official portal to complete your application.
- Ensure your resume and cover letter are tailored to the job description using our AI tools.
Frequently Asked Questions
Who is hiring?▼
This role is with ThoughtsWin in Surrey.
Is this a remote position?▼
This appears to be an on-site role in Surrey.
What is the hiring process?▼
After you click "Apply Now", you will be redirected to the employer's official site to submit your resume. You can typically expect to hear back within 1-2 weeks if shortlisted.
How can I improve my application?▼
Tailor your resume to the specific job description. You can use our free Resume Analyzer to see how well you match the requirements.
What skills are needed?▼
Refer to the "Job Description" section above for a detailed list of required and preferred qualifications.