The Python Oracle

Convert columns into rows with Pandas

Become part of the top 3% of the developers by applying to Toptal https://topt.al/25cXVn

--

Track title: CC P Beethoven - Piano Sonata No 2 in A

--

Chapters
00:00 Question
00:48 Accepted answer (Score 390)
02:11 Answer 2 (Score 30)
02:38 Answer 3 (Score 10)
03:22 Answer 4 (Score 8)
03:46 Thank you

--

Full question
https://stackoverflow.com/questions/2865...

Accepted answer links:
[melt]: https://pandas.pydata.org/docs/reference...
[pd.melt]: http://pandas.pydata.org/pandas-docs/sta...
[has been deprecated]: https://stackoverflow.com/questions/4412...
[pd.DataFrame.sort_values]: https://pandas.pydata.org/pandas-docs/st...

Answer 2 links:
[set_index]: http://pandas.pydata.org/pandas-docs/sta...
[stack]: http://pandas.pydata.org/pandas-docs/sta...
[reset_index]: http://pandas.pydata.org/pandas-docs/sta...

Answer 3 links:
[pd.wide_to_long]: http://pandas.pydata.org/pandas-docs/sta...
[pd.melt]: http://pandas.pydata.org/pandas-docs/sta...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#python #pandas

#avk47



ACCEPTED ANSWER

Score 421


Use .melt:

df.melt(id_vars=["location", "name"],
        var_name="Date",
        value_name="Value")

  location    name        Date  Value
0        A  "test"    Jan-2010     12
1        B   "foo"    Jan-2010     18
2        A  "test"    Feb-2010     20
3        B   "foo"    Feb-2010     20
4        A  "test"  March-2010     30
5        B   "foo"  March-2010     25

Old(er) versions: <0.20

You can use pd.melt to get most of the way there, and then sort:

>>> df
  location  name  Jan-2010  Feb-2010  March-2010
0        A  test        12        20          30
1        B   foo        18        20          25
>>> df2 = pd.melt(df,
                  id_vars=["location", "name"], 
                  var_name="Date",
                  value_name="Value")
>>> df2
  location  name        Date  Value
0        A  test    Jan-2010     12
1        B   foo    Jan-2010     18
2        A  test    Feb-2010     20
3        B   foo    Feb-2010     20
4        A  test  March-2010     30
5        B   foo  March-2010     25
>>> df2 = df2.sort(["location", "name"])
>>> df2
  location  name        Date  Value
0        A  test    Jan-2010     12
2        A  test    Feb-2010     20
4        A  test  March-2010     30
1        B   foo    Jan-2010     18
3        B   foo    Feb-2010     20
5        B   foo  March-2010     25

(Might want to throw in a .reset_index(drop=True), just to keep the output clean.)

Note: pd.DataFrame.sort has been deprecated in favour of pd.DataFrame.sort_values.




ANSWER 2

Score 30


Use set_index with stack for MultiIndex Series, then for DataFrame add reset_index with rename:

df1 = (df.set_index(["location", "name"])
         .stack()
         .reset_index(name='Value')
         .rename(columns={'level_2':'Date'}))
print (df1)
  location  name        Date  Value
0        A  test    Jan-2010     12
1        A  test    Feb-2010     20
2        A  test  March-2010     30
3        B   foo    Jan-2010     18
4        B   foo    Feb-2010     20
5        B   foo  March-2010     25



ANSWER 3

Score 10


pd.wide_to_long

You can add a prefix to your year columns and then feed directly to pd.wide_to_long. I won't pretend this is efficient, but it may in certain situations be more convenient than pd.melt, e.g. when your columns already have an appropriate prefix.

df.columns = np.hstack((df.columns[:2], df.columns[2:].map(lambda x: f'Value{x}')))

res = pd.wide_to_long(df, stubnames=['Value'], i='name', j='Date').reset_index()\
        .sort_values(['location', 'name'])

print(res)

   name        Date location  Value
0  test    Jan-2010        A     12
2  test    Feb-2010        A     20
4  test  March-2010        A     30
1   foo    Jan-2010        B     18
3   foo    Feb-2010        B     20
5   foo  March-2010        B     25



ANSWER 4

Score 8


I guess I found a simpler solution

temp1 = pd.melt(df1, id_vars=["location"], var_name='Date', value_name='Value')
temp2 = pd.melt(df1, id_vars=["name"], var_name='Date', value_name='Value')

Concat whole temp1 with temp2's column name

temp1['new_column'] = temp2['name']

You now have what you asked for.