Why does fillna with median on dataframe still leaves Na/NaN in pandas?

--------------------------------------------------
Hire the world's top talent on demand or became one of them at Toptal: https://topt.al/25cXVn
and get $2,000 discount on your first invoice
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Riding Sky Waves v001

--

Chapters
00:00 Why Does Fillna With Median On Dataframe Still Leaves Na/Nan In Pandas?
01:43 Accepted Answer Score 5
02:11 Thank you

--

Full question
https://stackoverflow.com/questions/5025...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#python #pandas #dataframe #series #imputation

#avk47

ACCEPTED ANSWER

Score 5

The problem is with this line:

TT_df = TT_df.fillna(TT_df.median())

Your dataframe has strings and you are attempting to calculate medians on strings. This doesn't work.

Here's a minimal example:

import pandas as pd, numpy as np

df = pd.DataFrame({'A': ['A', 'B', np.nan, 'B']})

df = df.fillna(df.median())

print(df)

     A
0    A
1    B
2  NaN
3    B

What you should do is fillna with median only for numeric columns:

for col in df.select_dtypes(include=np.number):
    df[col] = df[col].fillna(df[col].median())