All articles Business

AI for Machine-Based Health Diagnosis Part 1

Part 1: Introduction to Terms, Core AI Concepts


This is the first in a multi-part series on applied Artificial Intelligence for Health Care Diagnosis. This initial part covers the basic terms and concepts that are important building blocks for understanding the more recent advances in Artificial Intelligence and how they are being combined, optimized and applied to the real-world challenges of the health care needs globally.


The term “artificial intelligence” (AI) dates back more than 60 years to the mid 1950’s. It refers to what appears to be learned intelligence by computers. This contrasts to natural intelligence found in humans and animals. In that case, intelligence is both learned and genetically inherited. The learning process for humans has a formal and social component. For humans and animals, learning is an essential ingredient of survival. They must absorb and comprehend information about their environment and surroundings, make decisions and adapt as needed for survival. For AI, this is much more of a force-feeding process of information and learning into a mathematical representation.

The science of Artificial Intelligence relies on several disciplines, approaches and techniques over the past 60 years. The application of these techniques varies tremendously depending on what area of interest is being addressed, the state of computational speed and algorithmic efficiency, and of course the availability of data. Research in the Artificial Intelligence area is vibrant, ongoing and an area of significant investment. New techniques, algorithms and approaches are constantly being introduced and refined. Rarely does one singular technique solve "big problems" like patient diagnosis, but rather the creative combination plays a bigger role in reality.

Historically we have seen many “false dawns” for Artificial Intelligence over the past 60+ years. Expectations for how quickly AI can advance and meet true, real-world needs has constantly run decades ahead of reality. Given the start of commercial acceptance and deployment of AI today, however, we believe we have finally entered the perfect storm for the adoption of Artificial Intelligence in healthcare:

  • The cost of healthcare has been rising quicker than income and GDP growth in most countries; 2017 global healthcare spending grew by an estimated 7.9%
  • According to a 2017 report by the World Bank and the World Health Organization, half of the world population cannot obtain essential health services; this has a very significant impact on economic wealth and political stability
  • Two-thirds of the cost of healthcare is the labor costs; increases in productivity and efficiency in the labor force can have a tremendous impact on cost containment
  • The population in many countries is maturing, increasing the health care burden
  • Computing has become dramatically more powerful, less expense, highly interconnected by the Internet and accessible through smart phone globally
  • Advances in AI – from radical improvements in algorithmic efficiency to the application of new learning techniques have made capabilities more aligned with real-world needs.

For the foreseeable future, Artificial Intelligence is likely to be used to boost the efficiency and productivity of the doctors and the medical staff in the healthcare industry rather than replace them. The combination of human intelligence, instinct and reasoning combined with machine intelligence can increase performance significantly. AI can powerfully assist in gathering and structuring patient data and symptoms, can communicate that information in a concise and powerful way to the medical staff which can lead to higher productivity. As labor costs are the overwhelmingly largest component of medical costs in the healthcare industry, higher productivity effectively reduces per-unit labor costs, which might dampen the staggering increases in healthcare costs over the past several decades.

AI can also be delivered by interacting with low-cost, ubiquitous connected devices such as smart phones or voice-activated personal assistants (i.e. Alexa from Amazon). In areas of the world where they are dramatically underserved in terms of professional health care services (i.e. Rwanda), access to medical and wellness knowledge on a very personalized level can likely improve the quality of healthcare simply by providing the accessibility to it. Fears of the rise of the machines in healthcare are likely misplaced.



Early efforts to embed “intelligence” into computer software was based around what is known as “rule-based systems”. An example of such was usually an “expert system” which is built with carefully selected rule sets to solve a specific problem or set of problems within one subject area. Generally, a list of rules is gathered and when combined form a knowledge base. Action is taken based on information. This is referred to as an inference engine. While this approach is more flexible than a series of coded programming instructions that are executed, it still proved very limited when tackling real-world problems. The approach is very deterministic – “If X AND Y, then Z Action”. Over time, more rules and more exceptions must be added to refine the decision-making process, results and actions. This becomes unwieldy and unmanageable, and confidence overall in the accuracy of the decision-making process begins to diminish. In a situation where all possible inputs and possible outcomes can be very clearly identified in advance, incoming data is well structured, and there is a limited and discrete number situations or outcomes then a Rules-Based system can be helpful and effective.


Expert Systems are rules-based computer program built to mimic the thinking and reasoning process that a human being goes through to solve a specific problem or set of problems. As in our description above for Rules-Based Systems, expert systems are made up of a knowledge base and an inference engine. The knowledge is both factual (known from widely published materials, text, journals) and heuristic (from experience and judgement).

Like the shortfalls identified in Rule-Based systems in general, the expert system is only as good as the breadth, depth and accuracy of the knowledge base it has. This is generally very limited, and the expert system lacks any common sense or ability beyond its knowledge base and relies on more knowledge being fed to it for improved decision making. This makes rule-based systems inheritably restricted to the rules that have been programmed by an expert. New insights cannot be drawn based on the input data. When working with human experts in a subject matter area, humans often “skip steps” in articulating their process they go through for a situation. This is because they do things from prior experiential knowledge. The practical limits to Rules-Based Systems and Expert Systems were key catalysts for the development of learning-based systems in Artificial Intelligence.


 Due to the limits identified above in rules-based systems, AI research shifted to developing numerous techniques/approaches which allow a computer to learn without explicit programming logic but rather through observation of data and training. This introduces the concept of continuous learning and refinement. When a computer looks at an initial set of data, it may draw conclusions and develop a prediction capability that is weak but better than nothing. As it begins to observe billions of instances of data, it continues to refine it’s shaping of the required algorithms and increases the accuracy dramatically. Much like an athlete, the more training he undertakes, the better he becomes.

Of course, in areas where data is readily available in some structured digital form, the ability to apply machine learning is greater. In 2017, Stanford University introduced an algorithm called CheXnet that could analyze chest x-ray images and diagnose pneumonia on par with its human counterparts – radiologists. Because there were more than 100,000 chest x-ray images available, learning worked very well.

Unfortunately, this amount of very standardized data in a consistent format is more the exception than the norm in the healthcare industry. For a variety of reasons including disparate legacy computer systems, varying non-interoperable standards, and historic privacy challenges, it is difficult to help train machines in many areas in healthcare today.

Over the past 2 decades, many techniques or approaches to Machine Learning have been developed to attack different learning and data requirements. No one single algorithm or technique can be used to learn from all the various data and observations. A number of those are explained below. Often artificial intelligence researches experiment with different approaches and compare results, until the most efficient and accurate method to interpret the data is found.


Classification is an important part of machine learning. It is the process by which things are sorted and labeled. When machines pour through millions of pieces of data, they attempt to put data into discrete buckets to help better understand the shape of the data. Binary classification is categorizing data into only 2 discrete buckets e.g. “yes” and “no”. There are different types of classifications used based on the problem trying to be solved. Several of them are described in summary below. The following examples are known as “supervised learning” i.e. they require labelled data to be trained, similar to how a child learns in a supervised environment at school. It is hard to define how a human actually “thinks” and in reality, will be a combination of different methods. The ones below try to objectivity “thought” into smaller discrete concepts.


A Bayesian Network is a statistical graphical model which probabilistic relationship between two things such as a symptom and a disease. The graphical model represents a set of variables, such as disease, symptom and the conditional probabilities between them via a directed acyclic graph (DAG). A DAG in computer science is a finite directed graph where, if you start at point x and follow a route, it would not be impossible to route back to point x. This is important in a medical setting as we are interested in the interplay between symptoms and their relationship to disease, more than the relationship between symptoms alone. This forces the machine to use symptoms input by a user to drive towards a possible cause. The downside to a Bayesian network for healthcare is it requires a large amount of resources to ensure probabilities are accurate and suitable for each situation.


A decision tree is a simple means of classifying used in machine learning. Based on a series of criteria factors (“Is the person greater than 40?”, “Has the person actively smoked for the past 10 years?”), the logic branches/splits left or right on the criteria. Classifications get assigned through a series on criteria branches. These are often cumbersome to develop and hard to modify once produced. The more accurate/precise a decision tree model, often the more complex it is requiring to be, with more branches and further branches. These are often hard to modify, and scale once produced.


Linear regression has been around for a long time. Given two-dimensional space of X and Y, it attempts to understand and express the mathematical relationship between those dimensions. Given X, it predicts what Y is likely to be. This assumes that there is a linear and predictable pattern to the data. In the real world, there are many relationships that do not fit that criteria and many that do. For example, rainfall and crop yield is a simple form of a linear relationship. Of course, in reality it is not as simple as this as there are many variables to consider.  


Logistic regression is the appropriate classification to use when the output variable is binary i.e. 0 or 1, yes or no. This is a more sophisticated form of regression where the other variable does not need to be binary (ordinal, nominal etc.). For example, how does the probability of getting diabetes (yes or no) change for every addition Kg a patient is overweight and the average sugar intake per week. This form of regression requires a lot of labelled, and accurate data before it can make an accurate prediction.  


In the next article we discuss more complex forms of artificial intelligence including the basics of neural networks and deep learning.


Dr. Sunir Gohil is both an MD and a PhD in Data Science and is a member of the Clinical AI group at Babylon Health in London.

Brian E. Skiba is a member of the Babylon Health US team and is based in Austin, Texas.