Spring Boot (or Node.js, Django etc.) with Websocket on AWS

Please read on if you’re trying to implement a Spring Boot (or Node.js, Django etc.) Websocket-enabled application on AWS. I’ll keep this brief, covering only the high-level design of how multiple nodes of an AWS-based application cluster get notified of events, and in turn notify connected TCP clients. I’ll not be discussing how client connections to application cluster be maintained. For that, some good folks have already shared extensive information, like:

And if you’re still thinking about whether to use long-polling or Websocket for your interactive application, these might help sway it one way or the other:

Most of our services are created using Spring Boot, though we also use Node.js to a good extent. When it came to design a interactive financial order blotter with interactivity across user sessions, Spring Boot’s support for Websocket clinched the deal. Spring Boot supports Websocket using a Simple in-memory broker, and a full-featured broker for a clustered scenario (which is what we’re going for). Latter approach decouples message passing using a MQ broker like ActiveMQ, RabbitMQ etc. and works amazingly, that allows multiple nodes of a service to provide near real-time updates to connected clients, based on actions from the same group of clients – which is what a order blotter’s selling point is. For more information on how it’s enabled, please refer:

High-level glimpse of how this could be realized on AWS:

Websocket_MQBroker
Autoscaling app cluster with ActiveMQ network of brokers

It looks simple, but despite its many advantages there’re a few issues with this approach, specially when implementing on AWS:

  • MQ broker cluster is another piece of infrastructure to manage, monitor and upgrade continuously
  • Data replication and failover for DR scenario is not straightforward (required continuous back-up of EBS volumes, and then creating new EC2 instance with backed-up volume)
  • Broker network configuration can’t be static (again considering DR/Failover), and maintaining it dynamically is cumbersome – Though there’s a solution in case of RabbitMQ

There’re couple of great alternatives, while using the Simple in-memory broker feature of Spring Boot. And both options depend on AWS managed highly-available messaging services – SNS (pub-sub) and SQS (point-to-point).

Option 1 – Use SNS and SQS with tight coupling

Websocket_SQS-SNS
Autoscaling app cluster with SNS and SQS

In this approach, we’ve replaced the full-featured MQ broker with SNS and SQS. The high-level workflow is:

  • Create a SNS topic, where each service node could send events to (to enable Websocket communication with connected clients)
  • Create a node-specific SQS queue at application boot time, if doesn’t exist already (based on EC2 hostname/private IP), subscribing to the SNS topic
  • User A performs some action on service node A
  • Node A sends action event to SNS topic, and updates UI data for User A and other connected clients to that node via in-memory broker
  • SNS topic publishes action event to Nodes B and C, via respective SQS queues
  • Nodes B and C receive action event from respective SQS queues, and update UI data for respective connected clients via in-memory broker

If the service stack is Autoscaling or the nodes are ephemeral in some other manner, a process to clean-up unused SQS queues is required. A Lambda function, scheduled using Cloudwatch events works perfectly.

Option 2  – Use SNS and SQS with relatively-loose coupling

Websocket_SQS-SNS-DynamoDB
Autoscaling app cluster with DynamoDB, Lambda, SNS and SQS

In this architecture, application is not responsible to send an event to SNS directly. DynamoDB streams and Lambda function take care of that. If you’re already using or planning to use DynamoDB for persistence, this is a great option. The high-level workflow (with highlighted differences) is:

  • Enable stream(s) for DynamoDB table(s) of interest
  • Create a Filter Lambda function, which receives action events from DynamoDB stream(s)
  • Create a SNS topic, which could receive action events from Filter Lambda function
  • Create a node-specific SQS queue at application boot time, if doesn’t exist already (based on EC2 hostname/private IP), subscribing to the SNS topic
  • User A performs some action on service node A
  • Node A modifies data in DynamoDB, and updates UI data for User A and other connected clients to that node via in-memory broker
  • DynamoDB stream sends action event to Filter Lambda function, which can be used to filter events for Websocket communication
  • Filter Lambda sends action event to SNS topic
  • SNS topic publishes action event to Nodes B and C, via respective SQS queues
  • Nodes B and C receive action event from respective SQS queues, and update UI data for respective connected clients via in-memory broker

Both of these options help mitigate challenges with full-featured broker approach, but the downside is that you lock-in to cloud provider managed services. Go-ahead depends on architecture guidelines at one’s place.

These alternatives are not for Spring Boot services only. These would work well with a clustered Node.js or Django Websocket-enabled application as well. Please do comment if you have used so, or could think of other possible design options.

5 thoughts on “Spring Boot (or Node.js, Django etc.) with Websocket on AWS

  1. Wow, wonderful weblog layout! How long have you been blogging for? you made blogging look easy. The overall look of your site is great, as well as the content!

    Like

  2. Excellent weblog here! Additionally your site so much up very fast! What host are you the use of? Can I get your affiliate link to your host? I wish my site loaded up as quickly as yours lol

    Like

  3. hi!,I like your writing so so much! percentage we keep in touch extra about your post on AOL? I require an expert in this space to resolve my problem. Maybe that is you! Taking a look ahead to peer you.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s