Best Practice AWS Architecture for Magento

Recently I had the opportunity of migrating a world renowned Australian women’s fashion brand to Amazon Web Services (AWS). After going through a few trial and errors, I settled for the following setup which I consider the best practice setup on AWS for Magento 1 and 2:

  • 3 x AWS EC2 running Ubuntu 16.04 on different availability zones for the front-end
  • 1 x EC2 with Elastic IP acting as the admin instance
  • Auto-scaling with scale up action set for above 50% CPU usage, scale down below 30%
  • Media directory mounted via AWS EFS
  • Amazon Aurora for the database
  • Elastic Load Balancer
  • CloudFlare handling DNS and extra caching

This set up runs super smoothly. It’s a Magento 1 Enterprise site – the previous hosting had Varnish on it, but this runs better even without Varnish. I’ve set up sessions to be stored in the database.

AWS EFS really works a treat for the media directory. I don’t think you now have to bother with using extensions to use an S3 bucket or use a 3rd party NFS solution like S3FS.

Mounting the NFS drive via EFS is super simple. All you need to do is add something like the following to the root crontab:

Once you have your EC2 set up the way you wanted with the SSL certs etc. create the AMI and set up your auto-scaling. I love how it scales up and scales down seamlessly every day during peak and off-peak times. Those times usually have a pattern and is similar.

Magento AWS Architecture

Good for Magento 1 & Magento 2

Deployment happens via Git automatically for any commits pushed to the master branch. After a few attempts with CodeDeploy, I figured all I needed was a cronjob running every 2 minutes doing a pull, followed by a post-merge git hook to clear the cache and full-page cache on each Magento instance.

Loader.io has once again come in handy with load testing. I have not tried to hit the server with a sudden DDoS-like traffic but under realistic, heavy traffic circumstances it was able to withstand it with no sweat. I’ve got it to scale between 3 to 12 servers (m3.large) but it hasn’t needed more than 5 yet. It would be interesting to really push this test, though unfortunately, the system is now live and it’ll be harder to do a test like that unless there is additional funding for that.

At the end of the day, the client is happy and I’m happy that the site is live with minimal interruptions. Now it’s a truly fault-tolerant system with much better security and theoretically near-unlimited scalability. The client now does not have to deal with server outages that used to happen very frequently.

By the way, have you heard that most of the Shopify stores were down during the massive worldwide DDoS attack that took out one of the major DNS servers? All my clients on AWS were happily still selling. Not that AWS has never gone down but, could this be the ultimate reason against a SaaS eCommerce model?

screen-shot-2016-10-21-at-11-32-26-pm

Comments

  • Franck Garnier

    March 2, 2017 at 2:41 am

    Do you face about some limitations with AWS EFS ?

    Because I read this information :

    “Amazon EFS uses a credit system to determine when file systems can burst. Each file system earns credits over time at a baseline rate that is determined by the size of the file system, and uses credits whenever it reads or writes data. The baseline rate is 50 MiB/s per TiB of storage (equivalently, 50 KiB/s per GiB of storage).”

    Many thanks,

  • Francis Kim

    March 4, 2017 at 5:23 pm

    Hi Franck,

    As long as you don’t use EFS for things like Magento cache and/or sessions, using it for media file storage is perfectly fine. You ideally want a CDN to minimise the hit on the EFS too.

  • Steven Liao

    May 2, 2017 at 3:52 pm

    Hi Francis,
    You mentioned: “1 x EC2 with Elastic IP acting as the admin instance”.
    Would like to know how admin access the admin ec2 instance from public internet?

    Many thanks,

  • Francis Kim

    May 2, 2017 at 3:56 pm

    Hi Steven, great question. With Magento you can set the default base URL as a subdomain, for example admin.f00bah.com – and this can act as the admin interface versus going through the load balancer to a front-facing interface.

  • John Greenfield

    May 31, 2017 at 1:28 pm

    Hey Francis, can you do a similar setup for my new magento website?

  • Jin Ho

    July 4, 2017 at 4:24 pm

    Hi Francis,
    If I use Aurora with read replicas for Magento 1, I have to put down specific configuration in local.xml?
    감사합니다

  • Francis Kim

    July 4, 2017 at 9:17 pm

    Hi Jin Ho,

    Yes, you are correct! You have to make sure that the Security Group settings are correct so that your EC2 can connect to Aurora.

    감사합니다~

  • shuvo

    August 7, 2017 at 3:57 am

    Hi,
    Hope You are well..I am currently working on kind of same project you setup in aws. The problem is I what would be the base url of magento as I am using multiple ec2 for front end. request are eventually going to the server which is base url server. Your help will be highly appreciated.

  • Francis Kim

    August 8, 2017 at 3:08 am

    Hi Shuvo,

    The base URL should be the same for all your EC2s. The Elastic Load Balancer will distribute your traffic between the EC2s.

  • Huey

    August 31, 2017 at 2:49 am

    Hi Francis,
    How does the Cloudfront differentiate between public facing EC2 and the admin EC2? Would the URL just be the same?

  • Francis Kim

    September 6, 2017 at 2:54 pm

    Hi Huey,

    You should set up the admin EC2 on a subdomain like admin.f00bar.com

  • Amit

    November 9, 2017 at 12:23 pm

    Hello Francis,
    Its a great blog! I am trying to setup a similar setup using EFS. You mentioned not “don’t use EFS for things like Magento cache and/or sessions, using it for media file storage is perfectly fine”. I am not very familiar with the magento caching. I thought /media folder had many cached images. Are you referring to this? I want to have a setup where EC2 instancing facing customers are in scaling. What is best way to manage Magento cache and/or sessions. I see lot of mention in other sites about ElastiCache etc… I don’t know what this is. What do you recommend for managing cache and/or sessions accross multiple EC2 instances?

  • Francis Kim

    November 14, 2017 at 11:40 am

    Elasticache is definitely one way (and a nice way) to do it. You can get around this by using sticky sessions on your ELB too.

  • awstechie

    March 14, 2018 at 7:26 am

    Hi Francis,

    Can you tell me how to setup magento2 admin in separate EC2 instances. I have update the “Use Custom Admin URL” of admin to a sub domain but after changing, admin page is automatically redirected to search page (home page)

  • Mahmoud Moharam

    May 3, 2018 at 9:11 pm

    3 x AWS EC2 running Ubuntu 16.04 on different availability zones for the front-end
    1 x EC2 with Elastic IP acting as the admin instance
    What is the specs of these EC2’s ??

  • Francis Kim

    May 10, 2018 at 7:29 pm

    I would recommend a t2.medium at a minimum. And you should generally go for compute optimised instances if going for more expensive EC2s.

Write a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.