How to upload files to Amazon s3 using NodeJs, Lambda and API Gateway

Uploading binary to the s3 bucket using Amazon Lambda and API Gateway can be tricky sometimes and I’m going to share with you how I was able to do that.

Working with Accounteer has been a blessing to me in terms of taking up challenges. It has enabled me to explore new technologies and how to make use of a lot of Amazon web services.

I had the challenge to create an endpoint that will allow Accounteer users to upload files to their new or existing invoices, offers and purchases on the main app via the API.

I started my research and went through the Amazon documentation and it was stated that users can upload binaries directly to s3 using Lambda but to my amazement, it’s difficult to do that because Amazon API gateway passthrough only allows JSON as Content-Type.

Let me start by showing you how I did it. If you will like to follow my steps you must be a registered user on Amazon and must have created a bucket on your s3.

After you have created your bucket, there are other few steps that will be needed and I’m going to show you.

Firstly, you will need to open your bucket, click on the properties then edit the policy and permissions. This will allow the user to read and write to the bucket.

Bucket Policy

{
“Sid”: “PublicReadGetObject”,
“Effect”: “Allow”,
“Principal”: “*”,
“Action”: “s3:GetObject”,
“Resource”: “arn:aws:s3:::bucket-name/*”
}

Bucket permission

<CORSConfiguration xmlns=”http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>Authorization</AllowedHeader>
</CORSRule>
</CORSConfiguration>

After setting the above configuration on your new or existing bucket, the next thing is to create our Node app that Lambda can use to handle the file upload.

One of the easiest ways I used to upload files to S3 using Lambda is to convert it to a base64 encoded string and pass it to the buffer and then to the s3 putObject method and it’s as simple as that.

Dependencies

• aws-sdk
• moment