The Orcawise Innovation Program is a (volunteer) 100-day work experience program for graduates and professionals that want to transition to a career in AI. (Apply here and also connect on LinkedIn here: https://linkedin.com/in/kevinneary)
At Orcawise, we believe in the transformative power of data, especially when it's harnessed in real-time from the vast expanse of the web. We are seeking a talented Data Engineer who specializes in web scraping to fuel our AI-driven marketing solutions with rich, dynamic data.
About the role
- Design, develop, and maintain robust web scraping systems to extract valuable data from diverse online sources.
- Collaborate with data scientists and analysts to understand data requirements and ensure the relevancy and quality of scraped data.
- Construct, test, and optimize data pipelines to process and store large datasets efficiently.
- Handle challenges related to web scraping such as CAPTCHAs, IP bans, and page structure changes.
- Continuously monitor and maintain scrapers to ensure uninterrupted data flow.
- Collaborate with the broader engineering team to integrate web data into Orcawise's core systems.
- Stay updated with the latest technologies and best practices in web scraping and data engineering.
About You
- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
- Solid experience in web scraping tools and frameworks such as Scrapy, Beautiful Soup, or Selenium.
- Proficiency in building and maintaining data pipelines, preferably with tools like Apache Kafka or Apache NiFi.
- Strong programming skills, particularly in Python or Java.
- Understanding of web protocols, asynchronous programming, and web page structures.
- Ability to navigate challenges associated with web scraping, from changing web page structures to rate limits.
- Strong analytical and problem-solving skills with a keen eye for detail.
- Excellent communication skills, capable of working collaboratively across multiple teams.
Our commitments to you
- Flexible hours with access to top mentors on your work experience journey.
- Access to top-tier tools and platforms, ensuring you can scrape and process data at scale.
- A culture that values innovation, ensuring you're always learning and growing.
- Opportunities for professional development and training in the evolving field of data engineering.
- A collaborative team environment, where your insights and expertise will be highly valued.
- Flexible work schedules and arrangements, ensuring a balanced and fulfilling work-life scenario.
- A role that is pivotal to the success and growth of Orcawise's cutting-edge AI solutions.