Direct image upload to S3 from a Rails application

Cloud storage is becoming a must nowdays. It does have many advantages and is cheaper than buying storage for your VPS.

Recently I was dealing with application that was intended to store images. We chose to use Amazon S3 storage.

Since I was building a Rails application I used Carrierwave in combination with FOG to store images on S3. While uploading was working good it was quite slow. This is because actually the upload happens 2 times:
1. Your image is first uploaded to your server
2. After successful upload FOG sends your file to the S3

And when you upload thousands of images the process can go for hours. I needed to speed up the process, that is why I decided that I need to upload my files directly to the S3 skipping the upload to my server.

Fortunately there is a nice gem that did the trick for me Carrierwave direct
Integration is qute easy if you use single file upload and there is a good example in the documentation. However I was using jQuery FileUpload plugin to allow multiple files upload so my use case is a little bit different.

In order to make it work I am using the standart direct upload form

%form{:action => @uploader.direct_fog_url, :method => "post", :enctype => "multipart/form-data"}
  %input{:name => "utf8", :type => "hidden"}
  %input{:type => "hidden", :name => "key", :value => @uploader.key}
  %input{:type => "hidden", :name => "AWSAccessKeyId", :value => @uploader.aws_access_key_id}
  %input{:type => "hidden", :name => "acl", :value => @uploader.acl}
  %input{:type => "hidden", :name => "success_action_redirect", :value => @uploader.success_action_redirect}
  %input{:type => "hidden", :name => "policy", :value => @uploader.policy}
  %input{:type => "hidden", :name => "signature", :value => @uploader.signature}
  %input{:name => "file", :type => "file"}

On file 'add' callback of the jQuery Fileuploader plugin I used ajax request to get new file key, policy and signature and replace the values in the upload form

$('#new_image_uploader').fileupload
      dataType: 'xml'
      autoUpload: true
      add: (e, data) ->
          $.ajax({
            url: "new_photo_url"
            type: 'POST',
            success: (data) ->
              $('form').find('input[name=key]').val(data.key)
              $('form').find('input[name=policy]').val(data.policy)
              $('form').find('input[name=signature]').val(data.signature)
          })

          data.submit()

My new photo action looks like the following:

  def create
    @photo = @photos.create(params[:photo])

    render :json => {
      :policy => s3_upload_policy_document,
      :signature => s3_upload_signature,
      :key => @photo.store_key,
    }
  end

Generating the document policy and signature:

    def s3_upload_policy_document
      return @policy if @policy
      ret = {"expiration" => 1.day.from_now.utc.xmlschema,
        "conditions" =>  [
          {"bucket" => 'bucketname'},
          ["starts-with", "$utf8", ""],
          ["starts-with", "$key", @photo.image.store_dir],
          {"acl" => "private"},
          {"success_action_status" => "201"}
        ]
      }
      @policy = Base64.encode64(ret.to_json).gsub(/\n/,'')
    end
    def s3_upload_signature
      signature = Base64.encode64(OpenSSL::HMAC.digest(OpenSSL::Digest::Digest.new('sha1'), 'your_access_key', s3_upload_policy_document)).gsub("\n","")

      signature
    end

Note that you need the success_action_status field in your form and it needs to match your generated policy action status. Also your generated policy acl must match your form acl value.

The actual upload is handled by jQuery FileUpload plugin and you can track upload progress and use all the nice features that this plugin provide.


Zlatan Zlatanov

About Zlatan Zlatanov

A developer who love learning new things, living and working at the shore of Black sea.

  • Writing from Varna, Bulgaria

comments powered by Disqus