The 10 Best Live Chat Apps for Faster Customer Support

by Grant Gipe in


You're trying to buy shoes online, wondering if they'd feel as nice as they look, when ding a smiling face pops up on the bottom of the page asking if it can be of any assistance. You reply, and your worries are quickly assuaged by a real, live human.

That's the magic of live chat. Online stores can be lonely, impersonal places, where packages arrive in the mail before support emails get answered. Adding the digital equivalent of a salesperson wishing website visitors a good day can be the difference between a happy customer and another sale lost to competitors.

Live chat makes it easy to help customers faster, and prevents a backlog of delayed answers, angry customers, and lost opportunities. But it can also be a burden on your support team, something that's always interrupting their day with yet another question to be answered right now. That's why you need a great live chat app, one that helps you answer questions quicker and pause the incoming chats when your team's away.

Here are 10 of the best live chat apps to help your team support customers in real time.

Why Live Chat?

Like email, live chat lets you answer questions. Unlike email, however, live chat lets you engage customers while they’re still on your site and ready to make a decision—not hours after they’ve left.

Email lets you answer questions anytime, which is great for your support team. It might not be so great for your users.

Forrester Research found that 44% of online shoppers say that live chat is one of the most important features a site can offer, while Wells Fargo saw double-digit sales increase after adding live chat to their site.

Live chat works for customer support because it's fast and personal. The initial message asking if a customer needs help can be robotic, but the real answershave to come from a real human, right then without waiting days for an email response or hours for your call to be important enough for the company to answer it.

It's tough to offer great support over live chat. Your team will need to be available and ready to answer questions promptly during your listed business hours. And you'll need to be friendly enough to make the chat fun and engaging for your customers.

Set your customers' expectations well with listed hours for live chat and estimated wait time, though, and live chat support can help you answer questions faster, turn visitors into customers, and keep from having a support backlog each day.

The Best Live Chat Support Apps

Live chat requires a support team that's ready to answer questions, but it also requires a tool to help you chat with customers from your site. There's a ton of options, from live chat tools built into support center apps, to standalone chat tools that can integrate with your favorite support apps.

The best live chat apps for support also show which pages the customer has visited, provide canned responses to frequently asked questions, integrate with your other tools, and perhaps even include tools to show (rather than tell) a customer where to find what they’re looking for.

Let's look at the standalone live chat tools first, ones you can use just for chat alongside your other customer support apps. Then we'll peek at the best full-featured customer support apps that also include chat tools.

The Best Live Chat Apps

Just need a chat tool, for a simple way to customize a chat box, add it to your site, and start interacting with your customers in real-time? These apps are the best tools for just that. They're focused on chat, so they'll include the best features for talking to customers live—and then they'll integrate with the help desk and other apps you already use.

Zopim

for chatting with customers in any language—and on Facebook Messenger

 

Zendesk's one of the most popular customer support apps, designed around email but with the flexiblity to pull in other apps to help you offer support on social networks and more. That's how Zopim—a live chat app acquired by Zendesk in 2014—can help you support your customers in real-time, whether you already use Zendesk or not.

Zopim is a customizable support app with a modern design that can fit into any website. It integrates with CRM apps like Highrise and Salesforce, eCommerce apps like Magento and Shopify, and of course with Zendesk's helpdesk. It even integrates with Facebook Messenger, so you can answer questions from your fan page. Or, it can run just fine on its own, keeping track of each conversation with your customers, the pages they visit and more.

You won't have to wait for customers to reach out, either. Zopim visualizes all of your visitors based on categories so you can see who's sent a message, clicked the chat button (but hasn’t sent a message yet), activated a trigger, is currently browsing your site, or is stuck at the checkout page. From here, you can initiate chats with customers or view detailed information about a specific customer. Speed things up with shortcut canned messages, or let Zopim work for you with triggers that automatically start a chat with a customer when they hit a specific page.

It's most slick feature, though, is language translation. If you work with customers in different languages, Zopim will automatically translate messages for you. Any message you send in English will automatically be translated in the customer’s local language, and vice-versa. It's an almost magical tool for international customer support.

Zopim Pricing: Free for 1 agent; from $11.20/month per agent Basic plan for unlimited chats and 2 triggers

Olark

for fully customized live chat with cobrowsing

 

When you need live chat on its own, Olark is one of the most popular options thanks to its deep customization. You can tweak anything you want on the Olark chat box so it fits in with your branding. Add the text you want to the chat prompt, then use CSS to tweak the chat box colors, window size, position, and more.

Then, you can add it to your site, and integrate it even deeper using the Olark API to connect it to your store or apps. There's integrations with popular CRMs to pull in customer data, along with connections to link Olark with customer support apps like Desk and Highrise. You can even connect it to Google Analytics, to see where your customers visit before they start chatting.

But what if you don't want to offer chat throughout your whole site? Olark'sTargeted Chat feature is perfect for you. You can set up triggers to start chats automatically based on visitor data, or hide the chat box on certain pages or for those who haven't made a purchase yet. That way, you can offer live chat just to the customers most likely to buy—or those who have already bought your most expensive products.

Need to hold customers' hands during a chat? Olark's Cobrowsing feature displays your mouse cursor on the customer's screen, so you can highlight items and show them what to click. Or, you can control their screen if they let you, to help them make a purchase or fix a problem with far less explanation.

You need feedback from customers to know how to improve your support. At the end of a chat session in Olark, the customer is asked to rate the agent’s support and provide feedback. These results can then be tracked and displayed in reports to you know how your team is doing, as just one extra way Olark helps you make your live chat support the best possible.

Olark Pricing: Free for 1 agent with 20 chats/month; from $12/month Bronze plan for 1 agent with unlimited chats

Intercom Acquire

for lead tracking and stickers in live chat

 

Intercom may be best known as a marketing automation CRM, one that tracks what people have visited on your site and automatically follows up with drip emails. But it also includes a just-released live chat app that can stand on its own: Intercom Acquire.

Acquire is a two-way messaging platform: visitors can send a message to your support team, or you can use it to send messages to those visitors automatically. Using a team inbox, your support team is able to answer questions, and leave internal notes about customers' needs or concerns.

Like other chat tools, Intercom Acquire sits at the bottom of your site, ready for new messages. Its design, however, sets it apart, with a full-height chat pane that looks like part of your site or app. When visitors hover over the icon, they'll see a friendly chat pane that shows the photos of three online support agents. Chats themselves are fun, too, with stickers, emoji, and support for images and file attachments to both speed up support and make it feel like chatting with a friend. Or, if no one is online, Acquire lets them send an email to your support team, and gives an estimate of how quickly you'll get back in touch.

The best part about Acquire, though, is that it keeps track of everything about your visitors: their emails and chats, the pages they've visited and buttons they've clicked, and more. It can pull in data from Intercom, of course, but also connects with other CRMs and customer support apps. Whenever you're chatting with a customer, you'll have all their info at your fingertips and they’ll have all the information they need for the visitor and their history.

And you can let your whole team join in on live support, since Intercom is priced on how many people you talk to—not how many people are on your team.

Intercom Acquire Pricing: From $49/month for basic features and chats with up to 250 people

HappyFox Chat

for chatting with multiple customers at once

 

People don't like waiting in line for service—and there's no reason you shouldn't be able to help multiple people at once, especially if you're waiting on a customer to try a solution or find extra info. HappyFox Chat lets you chat with as many people at once as you can handle, with a simple interface that's focused just on your active chats.

There's still enough room for all the data you need, though. HappyFox Chat integrates with over 100 apps, so you can pull in support tickets from your help desk or recent order info from your eCommerce tool right inside chats with customers. Then, you can quickly reply with canned messages that can be activated by typing a hashtag. Everything's kept inside a chat window, showing you just the info you need without an overwhelming number of sidebars.

And it's free, too, for up to 10 agents—though your chat history is only kept for 14 days. That might be enough to help your team get started with live chat support, and then you can upgrade to a full plan later as your team grows.

HappyFox Chat Pricing: Free for 10 agents; from $14.99/month per agent Popular plan for 1 year of chat history and more integrations

SnapEngage

for PCI-encrypted and HIPPA compliant live chat

 

SnapEngage is a support chat app you've likely encountered before as a customer, as it's the tool brands like AirBNB and and T-Mobile use to power their chats. It's designed with enterprises in mind, with PCI encryption and HIPPA compliant data management so you can use it in any business.

It's customizable, integrates with popular apps, pulls in customer info from social networks, and includes mobile apps so you can chat on the go. It includes Cobrowsing as well, so you control your customer's browser during the chat if they let you, to help fix their problems and even fill out forms. Or, you can embed a secure form into your chat box, to collect credit card data or other private info securely right inside a chat, perhaps to close a sale or book an appointment.

And if you can't help a customer yourself, SnapEngage lets you hand the chat off to another support agent—or you can transfer the chat to a call and talk a customer through their problems without typing.

SnapEngage Pricing: $17/month Solo plan for 1 agent; from $60/month Basic plan for 4 agents and basic features

LiveChat

for live chat in desktop and mobile apps

 

Ever missed a chat session just because you accidentally closed a tab? LiveChat's desktop and mobile apps will keep that from happening again. Whenever someone starts chatting, you'll get a push notification and can quickly reply. Or, you can see everyone who's currently waiting for help, with color-coded icons to who who needs help the most urgently.

You'll never have to wait for customers to finish typing, as LiveChat shows what they're writing before they press send. And you'll also never have to look up their earlier chats—instead, just scroll up, and you'll see what was said last time your team talked to this customer.

LiveChat Pricing: $16/month per agent Starter plan for basic features and 60 day chat history

Pure Chat

for live chat without your own website

 

Don't have a website for your business—or need a way to offer live chat for mobile apps? Pure Chat makes that simple, with a customizable page where customers can start chats without your own website. In fact, the chat page can be your own simple website, complete with a customized background, contact and social info, and more—along with that all-important chat box.

Chatting with multiple customers at once is easy, with Pure Chat's tabbed interface that gives you multiple conversations and complete info on everyone at once. And if your customers would like a record of the chat once you've solved their problems, you can easily email them a transcript or share it with your team.

Pure Chat Pricing: Free for unlimited agents and 15 chats per month; $15/month Starter plan for 3 agents and unlimited chats

Alternative Live Chat Apps

The Best Help Desks with Live Chat

Live chat isn't all you need. You'll also need to answer your customers' emails, build a knowledge base with support documentation, and perhaps even jump on the phone with them to explain things quicker. If you want one app to do it all, these apps—or some of the above tools, including HappyFox, Intercom, and Zendesk—are some of the best ways to juggle everything support sends your way.

LiveAgent

for real-time insights on your chats

 

LiveAgent puts just as much focus on live chats as it does on email, and it shows. Instead of treating every ticket the same, LiveAgent puts chats in their own dashboard, complete with an overview of every currently ongoing chat. You'll see who's currently browsing your site, where they're from, and how long each chat has been going on.

If you'd like to nudge customers to get in touch if they need help, perhaps after they've been on your checkout page for 2 minutes or have visited your support site 3 times, LiveAgent can automatically prompt them to start chatting. But it'll also make sure your team won't get overwhelmed, as it'll only send out a prompt when there's a support agent free to chat. You can set how many chats each agent can handle at once, and LiveAgent will share the workload around, making sure no one gets overworked.

LiveAgent Pricing: From $29/month per agent Ticket+Chat plan for chat and support ticket tools

Zoho Support

for live chats and phone calls in one app

 

Already using Zoho tools to manage other parts of your business? Zoho Support might be the one extra app you need to start using to answer support emails, build a knowledge base, and chat with your customers.

It's deeply tied into Zoho's other apps, so you can pull in Zoho CRM data to learn more about your customers while you're chatting, along with the rest of the data you've already gathered in support. You can work together as a team, handing off tickets when they're too difficult for you or having chats routed to specific teams automatically based on the customer's questions.

And if you need to jump on the phone to help out a customer, Zoho Support's phone tools include everything you need. You can setup an interactive voice system to route calls, use voice mail to catch your customers' questions even when you're away, and call recording to keep a record of everything you've talked about.

Zoho Support Pricing: From $12/month per agent Professional plan for chat support

Freshdesk

for chatting with customers and your own team

 

One of the most popular full-featured customer support apps, Freshdesk includes live chat in addition to its email and knowledge base tools. And it's not just for supporting your customers—Freshdesk's live chat also lets you talk to anyone else on your team. That way, you can check with another support agent about how to answer a question—or just see how their day is going—without having to step away from your support app.

It'd be easier to answer customer questions if you knew more about their problems before you start chatting. Freshdesk makes that easy by including a form in the chat box to gather more info before initiating the chat session. It can then route the chat to the correct department or agent depending on the customer's needs.

Then, Freshdesk includes gamification features to help your team stay motivated. Chat with enough customers or answer enough tickets in a day, and you'll end up on the leaderboard with virtual trophies and badges. It's a tiny thing that might help keep you chatting with customers, even when you're tired of answering questions.

Freshdesk Pricing: From $25/month per agent Garden plan for live chat and email support tools

Source: https://zapier.com/blog/best-live-chat-cus...

A Beginner's Guide To Scaling To 11 Million+ Users On Amazon's AWS

by Grant Gipe


The Basics

  • AWS is in 12 regions around the world.

    • A Region is a physical location in the world where Amazon has multiple Availability Zones. There are regions in: North America; South America; Europe; Middle East; Africa; Asia Pacific.

    • An Availability Zone (AZ) is generally a single datacenter, though they can be constructed out of multiple datacenters.

    • Each AZ is separate enough that they have separate power and Internet connectivity.

    • The only connection between AZs is a low latency network. AZs can be 5 or 15 miles apart, for example. The network is fast enough that your application can act like all AZs are in the same datacenter.

    • Each Region has at least two Availability Zones. There are 32 AZs total.

    • Using AZs it’s possible to create a high availability architecture for your application.

    • At least 9 more Availability Zones and 4 more Regions are coming in 2016.

  • AWS has 53 edge locations around the world.

    • Edge locations are used by CloudFront, Amazon’s Content Distribution Network (CDN) and Route53, Amazon’s managed DNS server.

    • Edge locations enable users to access content with a very low latency no matter where they are in the world.

  • Building Block Services

    • AWS has created a number of services that use multiple AZs internally to be highly available and fault tolerant. Here is a list of what services are available where.

    • You can use these services in your application, for a fee, without having to worry about making them highly available yourself.

    • Some services that exist within an AZ: CloudFront, Route 53, S3, DynamoDB, Elastic Load Balancing, EFS, Lambda, SQS, SNS, SES, SWF.

    • A highly available architecture can be created using services even though they exist within a single AZ.

 

1 User

  • In this scenario you are the only user and you want to get a website running.

  • Your architecture will look something like:

    • Run on a single instance, maybe a type t2.micro. Instance types comprise varying combinations of CPU, memory, storage, and networking capacity and give you the flexibility to choose the appropriate mix of resources for your applications.

    • The one instance would run the entire web stack, for example: web app, database, management, etc.

    • Use Amazon Route 53 for the DNS.

    • Attach a single Elastic IP address to the instance.

    • Works great, for a while.

 

Vertical Scaling

  • You need a bigger box. Simplest approach to scaling is choose a larger instance type. Maybe a c4.8xlarge or m3.2xlarge, for example.

  • This approach is called vertical scaling.

  • Just stop your instance and choose a new instance type and you’re running with more power.

  • There is a wide mix of different hardware configurations to choose from. You can have a system with 244 gigs of RAM (2TB of RAM types are coming soon). Or one with 40 cores. There are High I/O instances, High CPU Instances, High storage instances.

  • Some Amazon services come with a Provisioned IOPS option to guarantee performance. The idea is you can perhaps use a smaller instance type for your service and make use of Amazon services like DynamoDB that can deliver scalable services so you don’t have to.

  • Vertical scaling has a big problem: there’s no failover, no redundancy. If the instance has a problem your website will die. All your eggs are in one basket.

  • Eventually a single instances can only get so big. You need to do something else.

 

Users > 10

  • Separate out a single host into multiple hosts

    • One host for the web site.

    • One host for the database. Run any database you want, but you are on the hook for the database administration.

    • Using separate hosts allows the web site and the database to be scaled independently of each other. Perhaps your database will need a bigger machine than your web site, for example.

  • Or instead of running your own database you could use a database service.

    • Are you a database admin? Do your really want to worry about backups? High availability? Patches? Operating systems?

    • A big advantage of using a service is you can have a multi Availability Zone database setup with a single click. You won’t have to worry about replication or any of that sort of thing. Your database will be highly available and reliable.

  • As you might imagine Amazon has several  fully managed database services to sell you:

    • Amazon RDS (Relational Database Service). There are many options: Microsoft SQL Server, Oracle, MySQL, PostgreSQL, MariaDB, Amazon Aurora.

    • Amazon DynamoDB. A NoSQL managed database.

    • Amazon Redshift. A petabyte scale data warehouse system.

  • More Amazon Aurora:

    • Automatic storage scaling up to 64TB. You no longer have to provision the storage for your data.

    • Up to 15 read read-replicas

    • Continuous (incremental) backups to S3.

    • 6-way replication across 3 AZs. This helps you handle failure.

    • MySQL compatible.

  • Start with a SQL database instead of a NoSQL database.

    • The suggestion is to start with a SQL database.

    • The technology is established.

    • There’s lots of existing code, communities, support groups, books, and tools.

    • You aren’t going to break a SQL database with your first 10 million users. Not even close. (unless your data is huge).

    • Clear patterns to scalability.

  • When might you need start with a NoSQL database?

    • If you need to store > 5 TB of data in year one or you have an incredibly data intensive workload.

    • Your application has super low-latency requirements.

    • You need really high throughput. You need to really tweak the IOs you are getting both on the reads and the writes.

    • You don’t have any relational data.

 

Users > 100

  • Use a separate host for the web tier.

  • Store the database on Amazon RDS. It takes care of everything.

  • That’s all you have to do.

 

Users > 1000

  • As architected your application has availability issues. If the host for your web service fails then your web site goes down.

  • So you need another web instance in another Availability Zone. That’s OK because the latency between the AZs is in the low single digit milliseconds, almost like they right next to each other.

  • You also need to a slave database to RDS that runs in another AZ. If there’s a problem with the master your application will automatically switch over to the slave. There are no application changes necessary on the failover because your application always uses the same endpoint.

  • An Elastic Load Balancer (ELB) is added to the configuration to load balance users between your two web host instances in the two AZs.

  • Elastic Load Balancer (ELB):

    • ELB is a highly available managed load balancer. The ELB exists in all AZs. It’s a single DNS endpoint for your application. Just put it in Route 53 and it will load balance across your web host instances.

    • The ELB has Health Checks that make sure traffic doesn’t flow to failed hosts.

    • It scales without your doing anything. If it sees additional traffic it scales behind the scenes both horizontally and vertically. You don’t have to manage it. As your applications scales so is the ELB.

 

Users > 10,000s - 100,000s

  • The previous configuration has 2 instances behind the ELB, in practice you can have 1000s of instances behind the ELB. This is horizontal scaling.

  • You’ll need to add more read replicas to the database, to RDS. This will take load off the write master.

  • Consider performance and efficiency by lightening the load off your web tier servers by moving some of the traffic elsewhere. Move static content in your web app to Amazon S3 and Amazon CloudFront. CloudFront is the Amazon’s CDN that stores your data in the 53 edge locations across the world.

  • Amazon S3 is an object base store.

    • It’s not like EBS, it’s not storage that’s attached to an EC2 instance, it’s an object store, not a block store.

    • It’s a great place to store static content, like javascript, css, images, videos. This sort of content does not need to sit on an EC2 instance.

    • Highly durable, 11 9’s of reliability.

    • Infinitely scalable, throw as much data as it as you want. Customers store multiple petabytes of data in S3.

    • Objects of up to 5TB in size are supported.

    • Encryption is supported. You can use Amazon’s encryption, your encryption, or an encryption service.

  • Amazon CloudFront  is cache for your content.

    • It caches content at the edge locations to provide your users the lowest latency access possible.

    • Without a CDN your users will experience higher latency access to your content. Your servers will also be under higher load as they are serving the content as well as handling the web requests.

    • One customer needed to serve content at 60 Gbps. The web tier didn’t even know that was going on, CloudFront handled it all.

  • You can also lighten the load by shifting session state off your web tier.

    • Store the session state in ElastiCache or DynamoDB.

    • This approach also sets your system up to support auto scaling in the future.

  • You can also lighten the load by caching data from your database into ElastiCache.

    • Your database doesn’t need to handle all the gets for data. A cache can handle a lot of that work and leaves the database to handle more important traffic.

  • Amazon DynamoDB - A managed NoSQL database

    • You provision the throughput you want. You dial up the read and write performance you want to pay for.

    • Supports fast, predictable performance.

    • Fully distributed and fault tolerant. It exists in multiple Availability Zones.

    • It’s a key-value store. JSON is supported.

    • Documents up to 400KB in size are supported.

  • Amazon Elasticache - a managed Memcached or Redis

    • Managing a memcached cluster isn’t making you more money so let Amazon do that for you. That’s the pitch.

    • The clusters are automatically scaled for you. It’s a self-healing infrastructure, if nodes fail new nodes are started automatically.

  • You can also lighten the load by shifting dynamic content to CloudFront.

    • A lot of people know CloudFront can handle static content, like files, but it can also handle some dynamic content. This topic is not discussed further in the talk, but here’s a link.

 

Auto Scaling

  • If you provision enough capacity to always handle your peak traffic load, Black Friday, for example, you are wasting money. It would be better to match compute power with demand. That’s what Auto Scaling let’s you do, the automatic resizing of compute clusters.

  • You can define the minimum and maximum size of your pools. As a user you decide what’s the smallest number of instances in your cluster and the largest number of instances.

  • CloudWatch is a management service that’s embedded into all applications.

    • CloudWatch events drive scaling.

    • Are you going to scale on CPU utilization? Are you going to scale on latency? On network traffic?

    • You can also push your own custom metrics into CloudWatch. If you want to scale on something application specific you can push that metric into CloudWatch and then tell Auto Scaling you want to scale on that metric.

 

Users > 500,000+

  • The addition from the previous configuration is auto scaling groups are added to the web tier. The auto scaling group includes the two AZs, but can expand to 3 AZs, up to as many as are in the same region. Instances can pop up in multiple AZs not just for scalability, but for availability.

  • The example has 3 web tier instances in each AZ, but it could be thousands of instances. You could say you want a minimum of 10 instances and a maximum of a 1000.

  • ElastiCache is used to offload popular reads from the database.

  • DynamoDB is used to offload Session data.

  • You need to add monitoring, metrics and logging.

    • Host level metrics. Look at a single CPU instance within an autoscaling group and figure out what’s going wrong.

    • Aggregate level metrics. Look at metrics on the Elastic Load Balancer to get feel for performance of the entire set of instances.

    • Log analysis. Look at what the application is telling you using CloudWatch logs.CloudTrail helps you analyze and manage logs.

    • External Site Performance. Know what your customers are seeing as end users. Use a service like New Relic or Pingdom.

  • You need to know what your customers are saying. Is their latency bad? Are they getting an error when they go to your web page?

  • Squeeze as much performance as you can from your configuration. Auto Scaling can help with that. You don’t want systems that are at 20% CPU utilization.

 

Automation

  • The infrastructure is getting big, it can scale to 1000s of instances. We have read replicas, we have horizontal scaling, but we need some automation to help manage it all, we don’t want to manage each individual instance.

  • There’s a hierarchy of automation tools.

    • Do it yourself: Amazon EC2, AWS CloudFormation.

    • Higher-level services: AWS Elastic Beanstalk, AWS OpsWorks

  • AWS Elastic Beanstalk: manages the infrastructure for your application automatically. It’s convenient but there’s not a lot of control.

  • AWS OpsWorks: an environment where you build your application in layers, you use Chef recipes to manage the layers of your application.

    • Also enables the ability to do Continuous Integration and deployment.

  • AWS CloudFormation: been around the longest.

    • Offers the most flexibility because it offers a templatized view of your stack. It can be used to build your entire stack or just components of the stack.

    • If you want to update your stack you update the Cloud Formation template it will update just that one piece of your application.

    • Lots of control, but less convenient.

  • AWS CodeDeploy: Deploys your code to a fleet of EC2 instances.

    • Can deploy to one or thousands of instances.

    • Code Deploy can point to an auto scaling configuration so code is deployed to a group of instances.

    • Can also be used in conjunction with Chef and Puppet.

 

Decouple Infrastructure

  • Use SOA/microservices.  Take components from your tiers and separate them out.Create separate services like when you separated the web tier from the database tier.

  • The individual services can then be scaled independently. This gives you a lot of flexibility for scaling and high availability.

  • SOA is a key component of the architectures built by Amazon.

  • Loose coupling sets you free.

    • You can scale and fail components independently.

    • If a worker node fails in pulling work from SQS does it matter? No, just start another one. Things are going to fail, let’s build an architecture that handles failure.

    • Design everything as a black box.

    • Decouple interactions.

    • Favor services with built-in redundancy and scalability rather than building your own.

 

Don’t Reinvent The Wheel

  • Only invest in tasks that differentiate you as a business.

  • Amazon has a lot of services that are inherently fault tolerant because they span multiple AZs. For example: queuing, email, transcoding, search, databases, monitoring, metrics, logging, compute. You don’t have to build these yourself.

  • SQS: queueing service.

    • The first Amazon service offered.

    • It spans multiple AZs so it’s fault tolerant.

    • It’s scalable, secure, and simple.

    • Queuing can help your infrastructure by helping you pass messages between different components of your infrastructure.

    • Take for example a Photo CMS. The systems that collects the photos and processes them should be two different systems. They should be able to scale independently. They should be loosely coupled. Ingest a photo, put it in queue, and workers can pull photos off the queue and do something with them.

  • AWS Lambda: lets you run code without provisioning or managing servers.

    • Great tool for allowing you to decouple your application.

    • In the Photo CMS example Lambda can respond to S3 events so when a S3 file is added the Lambda function to process is automatically triggered.

    • We’ve done away with EC2. It scales out for you and there’s no OS to manage.

 

Users > 1,000,000+

  • Reaching a million users and above requires bits of all the previous points:

    • Multi-AZ

    • Elastic Load Balancing between tiers. Not just on the web tier, but also on the application tier, data tier, and any other tier you have.

    • Auto Scaling

    • Service Oriented Architecture

    • Serve Content Smartly with S3 and CloudFront

    • Put caching in front of the DB

    • Move state off the web tier.

  • Use Amazon SES to send email.

  • Use CloudWatch for monitoring.

 

Users > 10,000,000+

  • As we get bigger we’ll hit issues in the data tier. You will potentially start to run into issues with your database around contention with the write master, which basically means you can only send so much write traffic to one server.

  • How do you solve it?

    • Federation

    • Sharding

    • Moving some functionality to other types of DBs (NoSQL, graph, etc)

  • Federation - splitting into multiple DBs based on function

    • For example, create a Forums Database, a User Database, a Products Database. You might have had these in a single database before, now spread them out.

    • The different databases can be scaled independently of each other.

    • The downsides: you can’t do cross database queries; it delays getting to the next strategy, which is sharding.

  • Sharding -  splitting one dataset across multiple hosts

    • More complex at the application layer, but there’s no practical limit on scalability.

    • For example, in a Users Database ⅓ of the users might be sent to one shard, and the last third to another shard, and another shard to another third.

  • Moving some functionality to other types of DBs

    • Start thinking about a NoSQL database.

    • If you have data that doesn’t require complex joins, like say a leaderboard, rapid ingest of clickstream/log data, temporary data, hot tables, metadata/lookup tables, then consider moving it to a NoSQL database.

    • This means they can be scaled independently of each other.

 

Users > 11 Million

  • Scaling is an iterative process. As you get bigger there's always more you can do.

  • Fine tune your application.

  • More SOA of features/functionality.

  • Go from Multi-AZ to multi-region.

  • Start to build custom solutions to solve your particular problem that nobody has ever done before. If you need to serve a billion customers you may need custom solutions.

  • Deep analysis of your entire stack.

 

In Review

  • Use a multi-AZ infrastructure for reliability.

  • Make use of self-scaling services like ELB, S3, SQS, SNS, DynamoDB, etc.

  • Build in redundancy at every level. Scalability and redundancy are not two separate concepts, you can often do both at the same time.

  • Start with a traditional relational SQL database.

  • Cache data both inside and outside your infrastructure.

  • Use automation tools in your infrastructure.

  • Make sure you have good metrics/monitoring/logging in place. Make sure you are finding out what your customers experience with your application.

  • Split tiers into individual services (SOA) so they can scale and fail independently of each other.

  • Use Auto Scaling once you’re ready for it.

  • Don’t reinvent the wheel, use a managed service instead of coding your own, unless it’s absolutely necessary.

  • Move to NoSQL if and when it makes sense.

Source: http://highscalability.com/blog/2016/1/11/...