Data Engineer Lead - Python and PySpark

Data Engineer / Senior / Supervisor

Principal Architect IAM

Principal Software Engineer

Techno-Economic Modeling Specialist

Senior Data Engineer

ASC-Portfolio Evaluations Geoscientist - UR Assessment & Data Solution

ASC - Portfolio Evaluations Geoscientist - UR Assessment & Data Solution

Sr. Engineer (New Energy)

Software Engineer - Quality

Responsibilities:Data Ingestion and Extraction:Develop and optimize data ingestion pipelines to efficiently extract data from various sources e.g. databases, APIs, files etc.Ensure data quality and integrity through robust validation and cleansing processesData Transformation and Processing:Design and implement data transformation logic using Python and PySpark to convert raw data into a suitable format for analysisOptimise data processing pipelines for performance and scalabilityData Storage and Management:Select and implement appropriate data storage solutions (cloud-based data warehouses, data lakes, etc.) based on data volume and complexityDevelop strategies for data retention, backup, and recoveryETL/ELT Development:Build and maintain efficient ETL/ELT processes to move data between different systems and platformsOptimize ETL/ELT pipelines for performance and cost-effectivenessCloud Infrastructure:Leverage cloud-based data engineering services (AWS, GCP, Azure) to build scalable and cost-efficient data solutionsManage and maintain cloud-based data infrastructureData Modeling and Optimization:Design and implement data models to support business intelligence and analyticsOptimize data structures and indexes for query performanceCollaboration and Communication:Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and deliver actionable insightsEffectively communicate complex technical concepts to both technical and non-technical audiencesData Governance and Security:Implement data governance policies and procedures to ensure data quality, security, and complianceProtect sensitive data through appropriate security measuresRequirements:At least a bachelor’s degree in computer engineering, or equivalentAt least 10 years in Microservices, with at least 4 years of experience in data engineering and architecturePrevious work experiences in using major cloud platforms (AWS, Azure, GCP) and containerisation technologies (Docker, Kubernetes)Proficient in Python and PysparkProficient in SQL and database designKnowledge and understanding of data warehousing, data lakes and big data technologiesKnowledge of data modeling and ETL/ELT processesKnowledge of PostgreSQL and OracleStrong problem-solving and analytical skillsAble to work independently and as part of a teamLead and manage the development team including assigning tasks and monitoring progressPreferred Requirements:Experience in using data visualization tools e.g. Tableau, PowerBI

Role Description:The Data Engineering Senior Supervisor will lead a small team of data engineering professionals to design, build and operate high performance data centric products or solutions through the use of modern engineering practices and technologies.In this role, you will provide guidance to ensure the delivered data centric products or solutions are scalable, sustainable and robust.You will also lead team development and cross team relationships and delivery to advance the company's engineering delivery.Responsibilities:Lead a local or regional team of engineering professionals that design, develop, deploy and enhance new and existing data centric products or solutions through the use of modern platforms and technologies.Provide routine inputs and guidance to the data engineering team on business requirements analysis, technical specification's preparation, product delivery, testing and operation to support businesses strategic direction.Coach engineers, share relevant technical approaches, identify new trends, modern skills and present new methodologies within the data engineering community.Lead and develop a team, coach and make decisions related to talent management, hiring, performance, and disciplinary actions.Other duties as assignedRequirements:Minimum QualificationsBachelor's degree in a related field or equivalent experience.Minimum of four years of related work experience.Minimum of two years leading a team of Data Engineers.Preferred QualificationsConfirmed data engineering experience with data storage and management of large, heterogenous datasets, including formats, structures, and cataloging with such tools as Iceberg, Parquet, Avro, ORC, S3, HFDS, HIVE, Kudu or others.Experience leading Big Data engineering teams working in environments such as Hadoop and SparkExperience leading technology strategies with cloud Platforms including AWS, GCP or AzureExperience leading engineering teams working in streaming and stream integration or middleware platforms, tools, and architectures such as Kafka, Flink, JMS, or Kinesis.Confirmed knowledge of best practice patterns and how to apply them to modern data architectures, including data warehouses, data lakes, data mesh, hubs and associated capabilities including ingestion, governance, modeling, observability and more.Experience leveraging transformation and modeling tools, including SQL based transformation frameworks, orchestration and quality frameworks including dbt, Apache Nifi, Talend, AWS Glue, Airflow, Dagster, Great Expectations, Oozie and others.Strong programming knowledge of SQL, Python, R, Java, Scala or equivalent.Proficiency in engineering tooling including docker, git, and container orchestration services.Strong experience leading devops teams with demonstratable understanding of associated best practices for code management, continuous integration, and deployment strategies.Experience and knowledge of data governance considerations including quality, privacy, security associated implications for data product development and consumption.A passion for quality and continuous improvement with an owners mindset.Ability to work effectively as part of a team, group and cultureAbility to navigate ambiguity and work in agile ways.

Responsibilities: Engage with multiple teams of highly talented engineers and architects to design and develop highly scalable, secure and configurable identity solutions that meet the needs of our customers and stakeholders. Actively engage with leadership across Product, Architecture, Design and Engineering to design, propose, discuss and refine new features and functionality. Collaborate with other engineering teams, IT, and business stakeholders to understand requirements and deliver innovative identity solutions that meet organizational needs. Communicate with stakeholders on technical standards, principles, methodologies and trends. Enhance overall system operational characteristics by establishing best practices for code quality, reduction in tech debt, building frameworks or shared techniques to improve engineering effectiveness. Identify opportunities for refactoring, API changes, and other types of tech debt remediation. Influence management for driving technical decision-making for the right selection of technology by giving forthright, understandable, and actionable advice. Work with standards defining organizations to understand the evolving standards, identify gaps, provide inputs on platform gaps, contribute to roadmap planning to build world class CIAM platform. Drive decision-making with team members with different views for simple to complex problems with the business requirements to help synthesize the best solution. Requirements: You must have significant business knowledge and technical expertise in application design and architecture, as well as knowledge of related domains of technical architectures (network, security, and infrastructure engineering), information architectures, and data architectures. Expert in back-end development using Java EE technologies (e.g. Java language, application servers, servlet containers, JMS, JPA, Spring MVC, Hibernate). In-depth knowledge of identity management technologies, protocols, and standards, and competent in designing solutions for reliability, availability, scalability, and performance. In-depth knowledge of Identity standards such as OAuth 2.0, Open ID Connect, SCIM etc. Must be a self- starter, work independently or as a team member, and have strong communication skills in interfacing with business partners and the Technology. Strong OOP skills, with ability to analyze requirements and transform into scalable software designs. Experience with HTTP, REST API's and competent designing and building web services/microservices in a commercial setting. Knowledge of SQL and other/NoSQL/modern database and storage technologies. Desired understanding of Oracle, JDBC and ORM frameworks (e.g. Hibernate). Experience with Design Patterns, MVC and frameworks (e.g. Spring). Competent in software engineering tools (e.g. Java build tools) and best practices (e.g. unit testing, test automation, continuous integration, etc.). Serve as a strong advocate of code craftsmanship, good coding standards and use of tools that will improve the quality of the delivered code Experience with using Agile/Scrum methodology for software development Deep understanding of web services and SOA related standards like REST/OAuth/JSON and SOAP/WSDL. Extra credit: Provide your GitHub account or code samples with your resume! 

The Role:We’re looking for a Principal Software Engineer that will be responsible for leading the administration, maintenance, and enhancement of our company's cloud infrastructure within our global data centers. This role demands experience and proficiency with Python and Ruby on Rails. The Engineer will ensure the reliability, security, and efficiency of our systems and networks while driving automation through innovation. Responsibilities:Develop global standards, process improvements, and promote best practices.Manage large-scale projects by coordinating with internal teams and third-party resources.Provide regular project updates to stakeholders through meetings.Lead the development and maintenance of internal automation scripts using Python and Ruby on Rails.Create comprehensive documentation for projects and workflows.Troubleshoot and resolve hardware issues on various enterprise servers (Blades, Storage, white label).Facilitate cross-functional communication with other Technical Operations groups.Participate in conference calls and meetings with vendors, suppliers, and team members.Collaborate closely with Data Center Engineers worldwide.Manage enterprise servers distributed globally across data centers.Participate in on-call rotations and be available for emergency situations.Assist with server provisioning and deployment.Send regular reports on server monitoring and other activities.Requirements:Familiarity with DevOps practices and CI/CD pipelines.Proficiency in shell scripting, specifically Python and Ruby on Rails.Strong knowledge of Linux and Windows operating systems.Expertise in computer hardware and resolving hardware issues on enterprise-level servers.Experience with installing various operating systems (Linux and Windows).Knowledge of cloud systems technologies, such as AWS.Strong verbal and written communication skills.Ability to prioritize tasks in a complex, fast-paced environment.Understanding of BIOS, BMC, out-of-band/lights-out server communication methods (IPMI, serial console).Experience with firmware management for all hardware components.Minimum of 5 years of relevant industry experience.Desirable Skills:Knowledge of data center best practices (fault tolerance, cable routing, etc.).Experience with heterogeneous server/network hardware failures and Linux OS/Junos.Experience in data center hardware deployments and building scalable infrastructure.Ability to adapt and drive change in a dynamic environment.A+ certification is a plus.

Responsibilities:Support strategic option analysis (e.g., make vs. buy, technology portfolio prioritization) via provision of cost-benefit assessments and analysis, as well as validation of economic and risk assumptions; present findings and recommendations based on analysis to senior management.Analyze trends and market intelligence to support the development of business cases (and value hypothesis) for technology projects going through the corporate technology stage-gate process.Create financial models and establish the value of technologies for potential commercial applications, in support of technology development, deployment and external licensing efforts. Support technology project teams in the development of a robust financial case for internally developed technologies, including techno-economic models for investment appraisal of technologies in the stage-gate process.Provide detailed financial and economic insights for evaluating technology project performance by analyzing relevant data against the technology scorecard KPIs.Support technology development, demonstration, internal deployment and external commercialization opportunities, including implementation of robust deployment plans. This includes tracking and accurately reporting the value realized from deployed technologies, as well as supporting internal and external reviews of the relevant financials.Provide input to help evaluate licensing opportunities to enable favorable license terms.Develop, manage and assure the quality of economic modelling tools used within TOC.Periodically review and improve financial models, maintain and update the repository of licenses and portfolio position on all approved technology/innovation applications.Work flexibly across TOC and/or R&D domains as required.Share and transfer techno-economic evaluation fundamentals knowledge to increase financial acumen in the organization. Requirements:Bachelor’s degree in Economics, Econometrics, Engineering, Statistics, Business Administration, or a related discipline. An advanced degree (MBA/Finance) is highly preferred.At least 15 years of professional experience, including a minimum of: 10 years in leading organizations in the energy or chemicals sector, and/or with leading management consulting firm(s); and five years’ conducting incremental, cost-benefit/effectiveness and budget impact analyses using mathematical, economic or statistical modelling tools and techniques, such as decision tree analysis, Markov modelling, and/or discrete event simulations, including Monte Carlo simulations and real options, risk adjustment, and risk management.In-depth knowledge and understanding of one or more relevant core businesses (downstream refining, petrochemicals, midstream, and upstream) and emerging technologies related to corporate sustainability (Circular carbon economy; CO2 capture, utilization and storage; nonmetallic materials; sustainable plastics; artificial intelligence; and industrial sensoring and monitoring).Ability to develop insights from market research to technology business case development; mastery in quantitative economic and financial analysis, portfolio modelling and technology valuation, including: incremental, risk-adjusted, cost-benefit, variance and scenario analyses, NPV, and sensitivity analysis, using a range of tools and applications (Microsoft Excel, SQL, SAS, R, MatLab, Tableau, and/or other VBA type software to analyze complex and large data sets, Planisware, or similar tools for portfolio and project management)

Responsibilities:Data engineers need to write code to develop, maintain, and optimize data pipelines, as well as to implement data storage solutions and data processing systems. They typically use programming languages such as Python, Java, or Scala, along with big data technologies like Hadoop, Spark, and KafkaDesigning scalable data architecture that supports both transactional and analytical workloads. This involves understanding business requirements and translating them into technical solutions.Developing robust data pipelines to ingest, cleanse, transform, and aggregate data from various sources (internal and external). This includes both batch and real-time/streaming data processing.Integrating data from disparate sources and ensuring data consistency, integrity, and reliability across different systems.Requirements:5-8 years experience with GCP, Big Query, Data Fusion, Analytics Hub and DataFlowExperience leading othersExperience managing projectsExcellent written and verbal communication skillsTime ManagementAbility to work alone and with a teamAbility to mentor junior membersA Bachelor’s degree in MIS or mathematics, statistics, or computer science, international equivalent, or equivalent job experience

Responsibilities:As a subject matter expert, you will perform the following;Geological technical assessments of resource plays to ensure prospectivity.Risk assessments to identify gaps in knowledge and quantify COS.Resource assessments to confirm value and support reserve estimation.Geologically support reservoir engineering with Commerciality Assessments.Strategic planning and monitoring to ensure the best plays are being de-risked and targets are being met.Work closely with the Exploration & Characterization Divisions to support planning and execution of wells.Evaluate current practices and identify opportunities for improvements.Effectively communicate through presentations and written reports to technical peers and management.Provide recommendations and advise in your particular area of expertise.Mentor younger staff and provide opportunities for knowledge sharing.Requirements:A Bachelor’s degree in Geology, Geophysics or related sciences; advanced degree is preferred.15 years of experience, with at least 5-10 years in unconventional plays, including shale or tight gas exploration, development, or strategic planning, across multiple unconventional hydrocarbon basins, is required.A solid background in the play-based approach to evaluate tight sand and shale plays, evaluation of risk and resource assessment within unconventionals, is essential.Experience managing and de-risking a portfolio of tight sand or shale opportunities from exploration through development, with a strong understanding of the geological and engineering factors that determine success within different unconventional plays, is essential.Knowledge in the integration of sedimentology, depositional environments, petrophysics, geomechanics, basic geophysics and basic reservoir engineering, is preferred.Experience integrating basic reservoir engineering and geology to tight and source rock reservoirs, is preferred.Previous experience working with reservoir engineers and reserve auditors to determine reserve and resource estimations, is preferred.Experience working with the production and drilling departments on well performance and optimization, is preferred.Experience working with Reservoir Engineering teams on Economics of Unconventional Field Development plans, is preferred.Candidates with more or less experience can potentially be considered, depending on the quality of their experience.

Responsibilities:Near field and wildcat exploration mapping and prospect generation throughout the Basins of Saudi Arabia.Well log interpretation and correlations using basic logs.Use of advanced log interpretations where available.Work effectively with petrophysical colleagues.Participate in reflection seismic data survey planning, 2D and 3D.Execute reflection seismic data interpretation 2D and 3D incorporating multiple surveys and vintages, onshore and marine, generating attributes as required. Well ties using synthetics and VSP.Interpret seismic, including seismic stratigraphic interpretations for subtle traps. Apply sequence stratigraphic techniques to identify subtle traps. Structural geology skills including fault interpretation and fault seal. Integrate diverse data-types to construct and promote prospects. Assess petroleum risk elements to identify viable prospects.Map making using industry standard tools.Prospect volume and risk estimates.Preparing and presenting professional-quality presentation materials to technical peers and management. Provide mentoring and technical assistance to young professionals.Requirements:Minimum of fifteen years of experience in subsurface exploration and a proven track record in prospect generation and maturation skills including broad international experience.Extensive experience in applying Clastic and Carbonate Sequence and Seismic Stratigraphy & Regional Geology using seismic and well data to analyze the subsurface at all scales relevant to petroleum systems, from the basin evolution, to the identification of stratigraphic traps and the timing of structural trap development, gross depositional environment reconstruction, reservoir potential, seal characterization and structural geology.Extensive experience in using applications relating to data interpretation, correlations, mapping and prospect generation e.g., Petrel, Petrosys.

Responsibilities:Renewables Department is seeking for a Renewables Power Engineering Specialist, with skills and competency to support the evaluation and development of renewables business cases and capital projects. Specific functions of the position are as follows:Develops long term master plans to develop new cost effective and innovative design strategies and solutions in different operating areas ofto meet sustainability goals.Conducts economic analysis and thoroughly evaluate different project alternatives to recommend best investment decisions with maximum attainable Net Present Value (NPV) for renewable systems.Examines technical studies to align with project proposal procedures. Such studies include equipment reliability, condition assessment, obsolescence, standard compliance, etc.Works with multidisciplinary internal and external organizations to provide technical guidance and recommendations on design details and construction documents as necessaryPerforms in solar resource data analysis for feasibility studies and system design (PR, Yield, Availability, Production etc.).Conducts site visits to perform for feasibility study including load profiling studies using power analyzerPerforms complex power and energy flow simulations to provide cost effective optimum Solar PV and energy storage systems size using load profile studies and operational constraints Responsible for understanding the operational requirement, conducting site visits, site feasibility & Renewables systems design and complete execution.Provides technical studies and guidance on energy storage technologies in terms cost, reliability, applicability, efficiency. Supports project development team to prepare technical scope of work, cost estimationReviews technical proposals, submittals and packages in line with international and  standardsDesigns both grid interactive or standalone PV systems for industrial applications with different installation configurations (rooftop and ground mounted) including balance of systems design calculations, layouts and schematic diagrams.Provides economic analysis (LCOE, NPV) and recommend best investment decisions with maximum attainable Net Present Value (NPV).Promotes a learning environment and continuous development. Acts as a mentor to junior engineers.Conducts seminars, presents case studies, and writes papers for peer-reviewed journals on engineering studies and projects he/she has overseen.The candidate will provide Technical evaluation for new energy startups, investment ideas and Pilot Projects. The candidate will Support the Company’s technology and energy ventures investment decisions inputs.Develops and maintains  Engineering Standards and Design Best Practices, Design Manuals or other documentation to enhance company profitability and productivity.Requirements:Bachelor's Degree in Electrical Engineering or Renewables Engineering. Master's Degree in Engineering is a preference.Five (5) to Ten (10) years’ engineering design experience in Solar PV, Concentrated solar and Energy storage (heat and chemical storage technologies).CEM certification is a plusStrong technical experience in Solar PV and Energy storage technologies (Flow batteries, Lithium batteries), system design, optimization, technology assessment and techno-economic feasibility studies, grid interconnection and reliability or distribution either with utility or end-users.Solar PV experience including plant design, installation and commissioning, system interconnection, codes and standards.Computerized Solar PV Systems analysis tools experience such as PVsyst, HOMER, AutoCAD, PVGIS tools, power analyzer/logger, IV curve tracing and battery testerExcellent command of spoken and written English.Excellent presentation and communication skillsAbility to work well within team environmentGreen Hydrogen, Fuel cells and Wind energy experience is a plus

Responsibilities:Work with SW teams to understand features and technical implementationWork with other engineers in designing, developing, and maintaining the automation infrastructureLead the charge in designing a testing strategy for each software productCreate and maintain testing plans for each software product that reflect customer usage models and approximate real-world environmentsPerform manual testing where required, whilst working to automate as much as possibleCreate scripts and tools to perform functional application, regression, and performance tests using various techniques (white and black box, etc)Provide standard metrics and results from all testing effortsBe an advocate for driving the software development process toward quality-centric methodologiesRequirements:Degree in Computer Science or related field or equivalent professional experienceDemonstrated testing skills and understanding of all aspects of QA processesStrong Linux command line skills, and the ability to use Linux effectively to create test environments and troubleshoot issuesSolid understanding of SW development lifecycleExperience with virtualization architecture and platformsFamiliarity with container-based software deployment and orchestration using Kubernetes is preferredExperience with CI/CD platforms and toolingExperience with testing frameworks like SeleniumCoding experience in python, bash, C, and/or javascriptGood troubleshooting and debugging skillsKnowledge of HPC clusters and/or networking a plus

Skills

Industry

Location

Jobtitle

20 co2 storage geoscientist Jobs found

Data Engineer Lead - Python and PySpark

Data Engineer / Senior / Supervisor

Principal Architect IAM

Principal Software Engineer

Techno-Economic Modeling Specialist

Senior Data Engineer

ASC-Portfolio Evaluations Geoscientist - UR Assessment & Data Solution

ASC - Portfolio Evaluations Geoscientist - UR Assessment & Data Solution

Sr. Engineer (New Energy)

Software Engineer - Quality

WhiteCrow
WhiteCrow is on a mission to connect companies with the best talent across the globe.

EMPLOYER

TALENT

COMPANY

Contact Us

JOBS BY SKILLS

JOBS BY FUNCTION

JOBS BY INDUSTRY

JOBS BY LOCATIONS

MODEL JOBS

JOBS BY CITY

Skills

Industry

Location

Jobtitle

20 co2 storage geoscientist Jobs found

Data Engineer Lead - Python and PySpark

Data Engineer / Senior / Supervisor

Principal Architect IAM

Principal Software Engineer

Techno-Economic Modeling Specialist

Senior Data Engineer

ASC-Portfolio Evaluations Geoscientist - UR Assessment & Data Solution

ASC - Portfolio Evaluations Geoscientist - UR Assessment & Data Solution

Sr. Engineer (New Energy)

Software Engineer - Quality

WhiteCrow WhiteCrow is on a mission to connect companies with the best talent across the globe.

EMPLOYER

TALENT

COMPANY

Contact Us

JOBS BY SKILLS

JOBS BY FUNCTION

JOBS BY INDUSTRY

JOBS BY LOCATIONS

MODEL JOBS

JOBS BY CITY

WhiteCrow
WhiteCrow is on a mission to connect companies with the best talent across the globe.