As the AI age unfolds around us, the dynamics of the job market are rapidly shifting. Professionals are increasingly concerned about job losses and displacements, while those entering the job market are apprehensive about their prospects. The fear of automation replacing routine tasks is prevalent among professionals, while new entrants fear a lack of opportunities or the necessity of acquiring entirely new skill sets. This fear is not unfounded. A recent estimate by the World Economic Forum predicts that artificial intelligence will replace approximately 85 million jobs by 2025. This change isn’t limited to any one sector.
As organisations across all industries strive to leverage data and AI to enhance productivity, improve processes, or deliver transformational value to their customers, the landscape of data-related professions is evolving rapidly.
The shrinking role of data scientists
Data scientists once hailed as the “sexiest job of the 21st century,” are witnessing a shift in their roles. The advent of AI and automation is reducing the traditional tasks performed by data scientists. Automation tools such as AutoML (automated machine learning) and low-code/no-code platforms have simplified data preparation processes, traditionally a significant portion of a data scientist’s job. Pre-built models available on cloud platforms like AWS, Azure, and GCP further streamline the model development process, once the domain of data scientists. As these tools become more accessible, the role of data scientists is becoming more strategic and specialised. They are now focused on sophisticated model creation and dealing with advanced algorithms like large language models (LLMs) and neural networks. Additionally, interpreting and communicating results, where translating complex data insights into actionable business strategies remains a critical skill that automated tools cannot easily replicate. Advanced research and development continue to rely heavily on the expertise of skilled data scientists to push the boundaries of AI and machine learning through innovative research.
Emergence of citizen data scientists
Instead, we are seeing the rise of “citizen data scientists”—individuals who do not have formal data science training but are able to perform data science work within an organisation. Typically, business analysts and other non-technical business users are increasingly able to perform such data-related tasks using user-friendly tools. This democratisation of data science means that many tasks traditionally handled by data scientists are now managed by professionals with less formal training in advanced analytics, further contributing to the shrinking traditional role of data scientists.
The expanding roles of data engineers and data architects
While the role of data scientists may be contracting, other data roles are experiencing significant growth. Data engineers and data architects are becoming increasingly vital in the AI age. The exponential growth of data and the complexity of managing it require robust data pipelines, scalable storage solutions, and efficient data integration processes. Data engineers are at the forefront of building and maintaining these systems, ensuring data quality and optimising workflows for performance and scalability.
Data architects, on the other hand, are responsible for designing the data infrastructure that supports these processes. They play a crucial role in defining data standards, ensuring data security, and establishing data governance protocols. As organisations adopt AI and cloud technologies, the expertise of data architects in managing complex data ecosystems becomes indispensable. Their ability to envision and implement comprehensive data management frameworks is key to leveraging new and innovative technologies effectively.
Critical skills for the data-first world
As the AI landscape evolves, professionals must focus on developing skills that are difficult to automate. Here are key skills for thriving in a data-first world:
Critical thinking and risk-taking: Professionals need to challenge the status quo and think creatively to solve complex problems. Embracing calculated risks and prioritising high-impact projects are essential for driving innovation.
Problem-solving: Breaking down complex business problems and finding effective solutions is critical. Structured problem-solving sets top data professionals apart in the AI age.
Storytelling and communication: The ability to translate complex data insights into actionable strategies is vital. Effective communication bridges the gap between technical teams and decision-makers.
Deep domain knowledge: Understanding the specific business context is crucial for connecting data insights to real-world challenges. Domain expertise allows data professionals to ask the right questions and provide relevant insights.
Technical depth: Mastering advanced data engineering techniques, AI algorithms, and the latest tools is essential. Continuous skill development ensures professionals remain competitive and can leverage emerging technologies effectively.
In my recent book, “Mastering the Data Paradox,” I propose a T-shaped capability-building model. This model combines a broad set of horizontal skills with deep vertical expertise in a specific area. As the AI age brings profound changes to the job market, particularly within data-related professions, adopting a T-shaped capability model can help data professionals navigate the complexities of the AI age and continue to drive value for their organisations.