How to write a file or data to an S3 object using boto3
--------------------------------------------------
Hire the world's top talent on demand or became one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------
Music by Eric Matyas
https://www.soundimage.org
Track title: Puzzling Curiosities
--
Chapters
00:00 How To Write A File Or Data To An S3 Object Using Boto3
00:35 Accepted Answer Score 386
01:21 Answer 2 Score 104
01:35 Answer 3 Score 47
02:00 Answer 4 Score 77
02:22 Thank you
--
Full question
https://stackoverflow.com/questions/4033...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #amazonwebservices #amazons3 #boto #boto3
#avk47
    Hire the world's top talent on demand or became one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------
Music by Eric Matyas
https://www.soundimage.org
Track title: Puzzling Curiosities
--
Chapters
00:00 How To Write A File Or Data To An S3 Object Using Boto3
00:35 Accepted Answer Score 386
01:21 Answer 2 Score 104
01:35 Answer 3 Score 47
02:00 Answer 4 Score 77
02:22 Thank you
--
Full question
https://stackoverflow.com/questions/4033...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #amazonwebservices #amazons3 #boto #boto3
#avk47
ACCEPTED ANSWER
Score 386
In boto 3, the 'Key.set_contents_from_' methods were replaced by
For example:
import boto3
some_binary_data = b'Here we have some data'
more_binary_data = b'Here we have some more data'
# Method 1: Object.put()
s3 = boto3.resource('s3')
object = s3.Object('my_bucket_name', 'my/key/including/filename.txt')
object.put(Body=some_binary_data)
# Method 2: Client.put_object()
client = boto3.client('s3')
client.put_object(Body=more_binary_data, Bucket='my_bucket_name', Key='my/key/including/anotherfilename.txt')
Alternatively, the binary data can come from reading a file, as described in the official docs comparing boto 2 and boto 3:
Storing Data
Storing data from a file, stream, or string is easy:
# Boto 2.x from boto.s3.key import Key key = Key('hello.txt') key.set_contents_from_file('/tmp/hello.txt') # Boto 3 s3.Object('mybucket', 'hello.txt').put(Body=open('/tmp/hello.txt', 'rb'))
ANSWER 2
Score 104
boto3 also has a method for uploading a file directly:
s3 = boto3.resource('s3')    
s3.Bucket('bucketname').upload_file('/local/file/here.txt','folder/sub/path/to/s3key')
http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Bucket.upload_file
ANSWER 3
Score 77
You no longer have to convert the contents to binary before writing to the file in S3. The following example creates a new text file (called newfile.txt) in an S3 bucket with string contents:
import boto3
s3 = boto3.resource(
    's3',
    region_name='us-east-1',
    aws_access_key_id=KEY_ID,
    aws_secret_access_key=ACCESS_KEY
)
content="String content to write to a new S3 file"
s3.Object('my-bucket-name', 'newfile.txt').put(Body=content)
ANSWER 4
Score 47
Here's a nice trick to read JSON from s3:
import json, boto3
s3 = boto3.resource("s3").Bucket("bucket")
json.load_s3 = lambda f: json.load(s3.Object(key=f).get()["Body"])
json.dump_s3 = lambda obj, f: s3.Object(key=f).put(Body=json.dumps(obj))
Now you can use json.load_s3 and json.dump_s3 with the same API as load and dump
data = {"test":0}
json.dump_s3(data, "key") # saves json to s3://bucket/key
data = json.load_s3("key") # read json from s3://bucket/key