Exploring a Healthcare Dataset:
The most important initial step when it comes to Clinical AI is understanding your data. Also known as Exploratory Data Analysis (EDA) – which entails looking at the size, shape, content, and issues of the given data set.
How do you explore the Healthcare DataSet:
The step involved in exploring the data set include:
1- Looking at the size of the dataset:
- How many rows (patients or entries) does it have?
- How many columns (features like BP, age, diagnosis) does each entry have? Columns are data points.
2- Preview the data:
- Look at the first 5-10 rows:
- Are there any missing values?
- Are columns named clearly?
(This gives you an overview of the data – more like a bigger picture).
3- Understand the Data Types:
- Different clinical questions require different kinds of answers -some clinical questions are best answered in numbers others own categories.
- Numbers (eg. Heart rate, glucose)?
- Categories (eg. Male/female, smoker/non-smoker)
- Dates (eg. Admission date, surgery date)
4- Check for missing values:
Missing data comes with a cost, it can lead to errors. So make sure:
- There aren’t fields with lots of empty or NULL entries.
EDA in Real -life:
Suppose you have a Sepsis Prediction Model deployed at your hospital, however, patient vitals aren’t being recorded with consistency, and as a result important data is missing from the data set on which the model works. The model would fail to predict sepsis as the model would think that the patients were doing fine, as it didn’t have the relevant data. So not having adequate amounts of appropriate data is a risk.
So the key questions to address when looking at data sets is:
- How was the data collected?
- Is the data complete?
- Is the data biased?
This brings us to the end of today blog post, but hopefully now, you would have an insight as to what really matters when it comes to data for a clinical AI model.
Leave a Reply