Job Description
Reporting
to the DataOps Engineering Lead,the position holder will play a pivotal
role in designing, implementing, and maintaining a robust data analytics
platform. You will work closely with data engineers, machine learning
engineers, data scientists, and software engineers to ensure seamless data
integration, processing, and analysis. This role requires a strong
understanding of data analytics principles, software engineering best
practices, and the ability to architect scalable and efficient data platforms.
Responsibilities
- Platform Architecture:
Design and develop a scalable and extensible data analytics platform to
support the organization’s data-driven initiatives. Architect data
pipelines, storage solutions, and analytics frameworks to handle large
volumes of data efficiently.
- Data visualization –
Creating visualizations such as charts, graphs, and dashboards to
communicate insights effectively to stakeholders.
- Data Integration and Processing: Implement data ingestion pipelines to integrate data from
various sources, including databases, data warehouses, APIs, and streaming
platforms. Develop ETL (Extract, Transform, Load) processes to preprocess
and clean raw data for analysis.
- Analytics Tools and Technologies: Evaluate, select, and integrate analytics tools and
technologies to support data exploration, visualization, and modeling.
Implement and optimize databases, data warehouses, and analytics
frameworks such as SQL, Hadoop, Spark, and Elasticsearch.
- Scalability and Performance: Optimize data processing pipelines and analytics workflows
for scalability, performance, and efficiency. Implement parallel
processing, distributed computing, and caching mechanisms to handle
large-scale data analytics workloads.
- Data Governance and Security: Ensure compliance with data governance policies, regulatory
requirements, and security best practices. Implement access controls,
encryption, and auditing mechanisms to protect sensitive data and ensure
data privacy and confidentiality.
- Monitoring and Maintenance: Develop monitoring and alerting systems to track platform
performance, data quality, and system health. Proactively identify and
resolve issues to minimize downtime and ensure uninterrupted data
analytics operations.
- Automation and DevOps:
Implement automation pipelines for infrastructure provisioning,
configuration management, and deployment. Establish continuous integration
and continuous deployment (CI/CD) processes to streamline platform
development and operations.
- Documentation and Training: Document platform architecture, data pipelines, and
analytics workflows. Provide training and support to data analysts and
data scientists to ensure effective use of the data analytics
platform.
Qualifications
- BS or MS in computer science or
equivalent practical experience
- At least 2-3 years of coding experience
in a non-university setting.
- Proficient understanding of distributed
computing principles
- Experience in collecting, storing,
processing and analyzing large volumes of data.
- Proficiency in understanding database
technologies
- Excellent written and verbal
communication skills
- Understanding of big data technologies:
Cloudera/Hortonworks
How To Apply