When is AI Worth it?
Consider Complexity, Cognition, and Critical Mass
Apr 28, 2021

Artificial Intelligence (AI) is the shiny new thing that everyone wants to say they have, or wants to sell to those who don’t. But, it’s important to recognize that AI is not always the right answer. Here, Software Engineer Sumeet Shah walks us through how to determine whether AI makes sense for your organization.

How do we define AI?

Before we delve into whether or not AI is right for you, let’s take a moment to define what we mean by “Artificial Intelligence.” This is easier said than done, since the definition of AI is a frequently debated topic. And it’s still evolving, as software that was considered AI a few years ago has been stripped of the title today. For our purposes, let’s borrow a definition from a recent AI paper from Harvard’s Berkman Klein Center for Internet and Society:

“Artificial intelligence (AI) systems are software (and possibly also hardware) systems designed by humans that, given a complex goal, act in the physical or digital dimension by perceiving their environment through data acquisition, interpreting the collected structured or unstructured data, reasoning on the knowledge, or processing the information, derived from this data and deciding the best action(s) to take to achieve the given goal. AI systems can either use symbolic rules or learn a numeric model, and they can also adapt their behaviour by analysing how the environment is affected by their previous actions.”

Building off of the key points in the definition above, we believe that AI is best for addressing problems that meet the 3 Cs – they are sufficiently complex, require cognition, and have critical mass.

Complexity

Because developing AI is resource intensive, you’ll want to make sure that the problem you’re trying to solve is complex enough to warrant the investment. Fortunately, complexity serves as a ‘gateway’ metric that can help filter out the problems that don’t need AI to be solved. One way to view problems is as “systems which are not in their desired states”. The goal of an AI agent working to solve this problem is to identify and possibly take actions (it may not always be empowered to act autonomously) to shift the system toward a favorable or winning state.

For example, a game of chess starts in a neutral state. As the game is played, both players will shift the game through its intermediate states and seek to end at a winning state where the enemy king is checkmated. We can quantify the complexity of a system by answering the following questions:

  • How many possible states does the system have? In a system described by a set of variables, each with a set of potential values, the number of states is the number of unique combinations of values. While this does technically mean that systems with continuous variables have an infinite number of possible states, the variables can often be discretized into significant buckets. Because it’s combinatorial, the complexity of a system grows rapidly as more variables are added.

    For illustration, a game of checkers on an 8 by 8 grid where each player has a total of 24 pieces is the most complex solved game to date, meaning that a win or draw can be guaranteed for a player if both players play optimally. A game of chess on the same sized board has only 8 more pieces (32 total). The addition of these pieces along with the pieces’ uniqueness and different move sets makes it so that chess has approximately 10 octillion times as many states as checkers.

  • How well can you perceive the state of the system? Some systems have variable sets that can be fully observed (like a game of chess), while others have variables that are hidden (like a game of cards where you can’t see your opponents’ hands). The less our ability to know the state, the more complex the problem becomes, as we must reason under more and more uncertainty.
  • How much control do you have over the state of the system? – This question addresses which variables you can affect, the degree to which you can affect them, and if you are the only one affecting them. A common example of a challenge where the AI has limited control is self-driving vehicles.

    While the AI simply has control over one vehicle, the system of traffic that the AI must navigate includes countless other vehicles acting independently and sometimes unpredictably, making it difficult for the self-driving vehicle to move safely. The more control you have over a system, the less complex your problem becomes. That being said, having full control over a system does not mean you’re dealing with a simple problem, as it may still be complex in other dimensions.

If your problem is complex enough that traditional methods are ineffective or prohibitively expensive, then AI might be the right choice for you. Complex systems that cannot be navigated with brute force can instead be approached with intelligence.

Cognition

AI systems are differentiated from more standard digital systems by their ability to process and reason with data in order to achieve a desired goal. Navigating through the countless states of a complex and dynamic system to reach the desired outcome requires the ability to think critically. Cognition can be thought of as the ability to manage the different types of complexity by reasoning effectively and efficiently.

  • Many States – In a system with an extremely high number of possible states, there may be several progressions that can lead from the starting state to a desired state, and often far more that lead to failure. If your problem requires being able to navigate a large state space, then AI is worth considering as it can determine which progressions are the most likely to lead to success, and which ones are best avoided.
  • Uncertain State – Deciding how to navigate a system with an obscured state requires being able to reason under uncertainty. This can involve inferring the values of hidden variables, making assumptions based on patterns or previous knowledge, or simply deciding the best course of action with an incomplete set of information.
  • Affecting State – Problems that require responding to and acting within a dynamic environment are often well-suited to AI. For the type of problem we’re looking at, the path from the starting state to the desired state is not immediately obvious, and a series of decisions must be made in an unstable environment.

    AI can learn not only what actions to take, but also how those actions may impact the system. It is able to make a series of decisions, where each choice has the potential to change the system in a way that affects the next choice. The ability to plan multiple moves in advance is a key benefit of AI, since the best paths to success may involve choices that are suboptimal in the short term, but more effective in the long run.

If you can handle your problem’s complexity with brute force solutions, then it’s usually cheaper and faster to do so. But, keep in mind that AI can often be effectively hybridized with brute force solutions. While a system solved with brute force can be navigated from memory, it requires that you have the memory available to store all of the data pertaining to the system’s potential states, which is not always feasible. A hybrid where the common paths are navigated from memory and AI is used to make decisions when in an unusual or uncertain state can be highly effective.

Critical Mass

If the scale of your problem is small and looks like it will stay that way for the foreseeable future, it’s almost certainly not worth developing AI to solve it. A significant chunk of the value of AI systems is the ability to deploy them at scale. Once the initial development cost has been paid, the more uses of your AI, the more value you’ll get out of your investment.

An application of AI that we’re all familiar with is Amazon’s product recommendation system. Products that we might like are recommended to us based on user profiles built up by observing our likes, dislikes, habits, demographic information, etc. This same functionality can be, and is, performed manually by smaller businesses with knowledge of their limited products and customers. It may not make sense for them to use AI for product suggestions, as they have already solved the problem through their customer relationships. For Amazon however, AI not only makes sense but is essential to serve its 150 million+ customers.

Conclusion

AI is exciting technology that’s being applied to solve some of the world’s most critical problems. But that doesn’t mean that it’s always the right solution. As you’re analyzing problems your business faces and how you should address them, remember the 3 Cs. AI becomes worth it when the problem you’re facing is sufficiently complex, requires cognition, and has critical mass.

Sumeet Shah is a Software Engineer at Asymmetrik specializing in building data management and visualization applications. He has helped develop Asymmetrik’s FHIR resources alongside numerous other projects. Some of his favorite technologies to work with are Angular, NodeJS, Keras, and MongoDB. You can read more by Sumeet here and here.