A Data Engineer (DE) in an small IT team, are mainly tasked with designing solutions that include data, data intelligence, data aggregation and reporting. The role involves understanding the disparate data sources in the organization that are on-premise/cloud bespoke/COTS systems, analyze the data schema and design the data elements that needs to be continuously extracted into a central repository for data analysis. They work closely with internal teams (graphic designers, UX Leads, business analysts, operations, business users) in the organization during the development life-cycle of an application. FSDs would need to ensure the applications are built with quality code, collaborate well with design elements, integrate with on-premise / cloud systems, bespoke / COTS applications, able to scale well with load, and perform efficiently.
Responsibilities
- Developing roadmap and architecture to manage data assets throughout the data lifecycle of the organization
- Managing and designing infrastructure for extraction, transformation, and loading of data from a wide range of data sources
- Liaise with the Data Scientists, Architects, software developers, and business analysts/users to understand how data needs to be converted, loaded, processed and presented
- Experience in designing data driven decisions backed through BI tools and Data
- Write technical design documentation
Technical Expertise
- 4+ years of experience as a data engineer/analyst or similar role.
- Experience in data ingestion, cleaning and processing tools.
- Design, develop and deploy programs, source codes, batch scripts, complex SQL stored procures, functions and triggers.
- Highly proficient in MS SQL Server, MS SQL Reporting Service (SSRS) and SQL Server Integration Services (SSIS)
- Experience in end-to-end design, coding, testing, review and implementation using Power BI or similar service offerings.
- Develop spreadsheets, diagrams and process map to document needs
- Familiarity with graphic design, data visualization and user experience .
- Experience with Python, JavaScript will be plus.
- Experience with database technology, RDBMS and No-SQL.
- Experience in performance optimization and tuning.
- Contribute effectively in a fast-paced, deadline-driven, and collaborative programming environment
- Stays up to date with current trends, best practices and new technologies
- Knowledge & practices of SDLC process and agile methodologies
Mindset & Behaviors
- Able to see the big picture, the vision and understands the business requirements and a strong passion for technology.
- Orientation to Detail, Communication, Teamwork, Time Management, Personal Responsibility, Future Orientation, Growth Mindset, Resilience
- A curious, creative problem solver that always want to learn
- Open-minded, and enjoy collaboration with both designers and other developers