boto encryption key with amazon s3

The two functions you probably mean are set_contents_from_filename and set_contents_from_file

If possible, I want to know, which is the key that is being used to encrypt the file.

The current server-side-encryption method is AES256 (Source), the key is generated on the server-side.

If encryption is set to true, the encryption takes place in server side right?

Yes, data is uploaded, then encrypted on the server side. If you wish, you can also encrypt the data on your client before uploading, but this would mean, for reading, you also have to decrypt it on the client. If you don't want to transfer data plain from and to the s3 servers, you can use the SSL endpoints

Encrypted, while downloading, the objects are decrypted in s3 and then start to download? Or does the decryption happen while downloading?

After uploading a file with the encryption header set, s3 will encrypt your file for storage and decrypt it when requested. The file is saved in a encrypted version on the physical storage

Workflow illustration from the AWS blog