In this short post, I describe how I configured Django to upload to Amazon S3 instead of a regular file-system upload. It can be useful for production scenarios.
We will be using the
django-storages package to make it easier to
upload to S3. It is always better not to reinvent the wheel, but if
you’d like to explore on how to do it on your own, you definitely
Install via pip.
I’m assuming that you already have a virtual environment set up for your django project, and are installing within it.
Now in my case, I’ve deployed my webapp to both Heroku and AWS. I have different settings files for both, and I wanted my S3 setup to be respected on both configurations. To this end, we create a separate settings file which holds the S3 settings.
1 2 3 4 5 6 7 8
In your production settings file, at the bottom, add an
if check for
production and import these settings. I do it by looking for a particular
environment variable which is set only on production.
1 2 3 4 5
S3BotoStorage, django-storages requires
the set of AWS credentials which you plan to use to upload to an S3 bucket.
If you don’t already have them, you can go to the IAM console on AWS
and generate them. These are the permissions I added to the security
group which I applied on these credentials -:
The primary purpose of these credentials is to allow (in my case) an admin user to upload/delete images on an S3 bucket. We will let the public view images, but not manipulate them in any other way, nor abuse the system. Note the IAM ID of the credentials, as you will need it later.
Now head to the S3 management console on AWS and create a bucket.
You will be presented with a prompt to enter the bucket name and the region where it should be deployed. Choose a region keeping in mind your target audience. You are also allowed to set up logging, with a prefix which is basically a ‘folder’ in which log files will be stored. (It is easier to think of it as a folder, though that is not entirely the case)
Once the bucket has been created, we have to configure certain permissions on it. This is what the Properties section of a newly created bucket looks like -:
Let’s add a permission policy to our bucket. Click on ‘Edit bucket policy’ and paste the following -:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
This is, in effect, a combination of policies. The first part of the policy enforces public-read, i.e., anyone can read data on the bucket. The second part of the policy allows any action to be performed (get, put, delete), but this is restricted to the user with the IAM ID as given. Paste the IAM ARN/ID from earlier here, and hit Save.
In my case, I had to edit the CORS configuration, though this may not be necessary for you. On the same Properties > Permissions section, hit the ‘Edit CORS Configuration’ button and paste the following -:
1 2 3 4 5 6 7 8 9 10
This allows users from other domains to make HTTP requests (GET and POST) on our bucket.
And there you have it. Now you can upload files to S3, and view them in
your bucket. There should be an
uploads folder (standard django stuff)
in the bucket after your first upload.