The Python Oracle

Random row selection in Pandas dataframe

Become part of the top 3% of the developers by applying to Toptal https://topt.al/25cXVn

--

Music by Eric Matyas
https://www.soundimage.org
Track title: Mysterious Puzzle

--

Chapters
00:00 Question
00:39 Accepted answer (Score 71)
01:00 Answer 2 (Score 380)
01:44 Answer 3 (Score 46)
02:24 Answer 4 (Score 12)
02:42 Thank you

--

Full question
https://stackoverflow.com/questions/1592...

Accepted answer links:
[has been deprecated]: https://pandas.pydata.org/pandas-docs/st...

Answer 2 links:
[method built-in]: http://pandas.pydata.org/pandas-docs/sta...

Answer 3 links:
[sample]: https://pandas.pydata.org/pandas-docs/st...
[pd.DataFrame.sample]: https://pandas.pydata.org/pandas-docs/st...
[np.ramdom.seed]: https://docs.scipy.org/doc/numpy-1.15.1/...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#python #pandas #random

#avk47



ANSWER 1

Score 413


With pandas version 0.16.1 and up, there is now a DataFrame.sample method built-in:

import pandas

df = pandas.DataFrame(pandas.np.random.random(100))

# Randomly sample 70% of your dataframe
df_percent = df.sample(frac=0.7)

# Randomly sample 7 elements from your dataframe
df_elements = df.sample(n=7)

For either approach above, you can get the rest of the rows by doing:

df_rest = df.loc[~df.index.isin(df_percent.index)]

Per Pedram's comment, if you would like to get reproducible samples, pass the random_state parameter.

df_percent = df.sample(frac=0.7, random_state=42)



ACCEPTED ANSWER

Score 72


Something like this?

import random

def some(x, n):
    return x.ix[random.sample(x.index, n)]

Note: As of Pandas v0.20.0, ix has been deprecated in favour of loc for label based indexing.




ANSWER 3

Score 12


The best way to do this is with the sample function from the random module,

import numpy as np
import pandas as pd
from random import sample

# given data frame df

# create random index
rindex =  np.array(sample(xrange(len(df)), 10))

# get 10 random rows from df
dfr = df.ix[rindex]



ANSWER 4

Score 5


Below line will randomly select n number of rows out of the total existing row numbers from the dataframe df without replacement.

df = df.take(np.random.permutation(len(df))[:n])