A recent CIO.com survey found that 64% of organizations face a shortage of data science skills and that lack is holding back their data exploration. Upskilling business analysts with advanced analytics capabilities is key to gaining the competitive edge that comes from exploring complex data sets, but knowing where to start and what tools will speed up this journey isn’t always clear.

We turned to the experts on our Scientific Advisory Board for their insights on the foundational skills necessary for data exploration, how network science and AI fit in, what ChatGPT will really replace in the world of work, and more.

For this discussion, we’re joined by:

• Professor Robert Berwick, MIT 
• Professor Adam Weirman, Caltech
• Professor Kostia Zuev, Caltech

Each of these experts brings valuable insights directly to our teams, helping them develop the kinds of solutions that have earned patents and solved complex challenges. We’re excited to share their thoughts on some of the most common questions in data and analytics today. 

What can business analysts who are new to advanced analytics do to start building up their data exploration skills?

Robert: Data exploration works best when you have someone who is sophisticated in statistics working with someone just starting out. It’s like an apprenticeship. Also, analysts should just start playing with the data because oftentimes that leads you to the right questions.

Kostia: Formulating the right question is a key foundational skill. If you want to do intelligent analysis, at minimum, you should know the right question to investigate and which data set to look at for answers.

Adam: Agree. If you get data exploration wrong, you’re looking at the wrong problem. You’re missing the point. It’s the absolute foundational first step to any analysis.

On the other side of that coin, can experts like data scientists benefit from data exploration tools?

Adam: Definitely. Having a tool that surfaces counterintuitive things you’re not going to see unless you were looking for it isn’t something that experts will have off the shelf. 

Kostia: Data scientists should also look at a tool—like Virtualitics—as a kind of MATLAB for data exploration. Use it to try out something quickly and see if it could work before you spend the time to program it yourself.

Can building a dashboard for reports achieve the same outcomes as data exploration?

Adam: Dashboards still rely on the user of the tool to do the exploration with their own insights and skill. Whereas, if you can expose those insights in an automated way, that’s big. Dashboards just can’t do that. Data exploration requires better tools. If you have easier tools for visualization and highlighting, you’re going to see a broader set of topics in more depth than if you just let intuition guide you to where you think connections in the data might exist.   

What does network science bring to the table for data exploration that wasn’t there otherwise?

Adam: As a data analyst, if you ignore network structure, you’re ignoring hundreds of important features connecting things in the data that could lead to new insights and conclusions. And most tools give you nothing. You miss all of that. And so having a tool that can let you visualize and explore the graph structure is crucial.

Speaking of automation, AI is top of mind in every business. What types of research are critical to the future adoption of AI?

Adam: Causality and safety. By that I mean, trustworthiness and interpretability. There’s a lot of racing ahead on “I can do an okay job for lots of questions and lots of topics,” but to make AI deployable in meaningful ways, it needs to do more than an “okay” job. 

Robert: Long-term cognitive research is needed about what instills trust. I agree that everything is racing ahead so fast now with AI and large language models (LLM) that no one is thinking about human-computer interaction like they used to. When they first designed the predecessor to the Macintosh, they did all this research at Bell Labs to figure out what was the cognitive load of moving a mouse and where should the trash can go to minimize the load on humans. They really thought about how to interface with the human side of it and instill trustworthiness. But I don’t really see this yet with AI. 

Will tools like ChatGPT replace data analysts and other jobs?

Adam: The more likely use in the near- and long-term is that it will make us more efficient at the things we’re doing. It won’t be that it isn’t working with human input. It’s working with human input to understand and target usage around how to help you get started, go faster, and be more careful. For example, I don’t think we’re far from a ChatGPT-like plug-in being trusted and used for math calculations. Then, the data exploratory piece becomes even more important and this piece will remain a human’s job. 

Kostia: AI will definitely take some jobs out of the market, especially routine ones, but everyone needs to learn how to use AI and ML solutions. It’s like how everyone needs to learn to use a computer now. And did the computer eliminate some jobs? Yes, but in many ways they created much more.

So, how do we prepare young professionals for a future where AI might be as commonplace as computers and cell phones?

Robert: We should train them all in the standard philosophy of what ethics means and what’s been learned about that. Because everyone can learn about the algorithms, but the ethical questions run deeper. 

Kostia: Yes, for business leaders, AI ethics is even more crucial to learn than the mathematical part. They may not design the algorithms themselves, but they have to guide groups of people to do this. And for that, having strong training in ethics philosophy is important.

Adam: People have this view that AI is magic. People don’t understand how it works, but it solves their problems. They don’t get the limitations of the various tools or understand that you’ll get terrible results when you apply them to the wrong places. Knowing when and when not to apply AI is key. Teaching those limitations, ethics being one of them, is essential.

A space where innovation can thrive

Our Scientific Advisory Board plays a key role in our innovative culture at Virtualitics. Our goal is always to build an environment where scientists and engineers can collaborate together, solving challenging problems that will deliver positive impacts across industries. 

Sound like the kind of team you’d like to be part of? Take a look at our current job openings.