Efficiently read big csv file by parts using Dask

Dask dataframe will partition the data for you, you don't need to use nrows/skip_rows

df = dd.read_csv(filename)

If you want to pick out a particular partition then you could use the partitions accessor

part = df.partitions[i]

However, you might also want to apply your functions in parallel.

df.map_partitions(process).to_csv("data.*.csv")