AWS SQS — In a NutShell

Joshua Callis
2 min readApr 17, 2023

Overview

  • SQS is a queue, e.g a Kinesis data streams producer can send messages to the SQS queue.
  • Consumer will poll messages from the queue.
  • Fully Managed.
  • Very low latency, less than 10ms!
  • No limit to how many messages can be in the queue.
  • Default retention is 4 days, but this can be increased up-to 14 days.
  • Can have duplicates messages (at least once delivery).
  • Offers best effort ordering, but no guarantees.
  • Messages have a limit of 256KB per message sent.
  • Messages are defined in the body and are of type string.
  • Can optionally provide a delay on delivery.
  • Response back once a message has been sent to SQS queue. This will contain a message identifier and a MD5 hash of the body.

Consuming Messages

  • Poll for messages, can do up-to 10 messages at a time.
  • Consumer will delete the message once it has been consumed via the message ID & receipt handle. Therefore, messages cannot be processed by multiple applications.

FIFI Queue

  • Names of the queue must end in .fifo
  • Can send up-to 3,000 message/s compared to a standard queue with batching enabled and 300 without.
  • Messages are processed in the exact order from the consumer that pushed them.
  • Messages are sent exactly once.
  • Can send large messages with SQS extended client (Java library). Producer will send the large payload 10GB to an s3 bucket and a small metadata message to the SQS queue, the consumer will then pull the metadata containing the bucket id to access the much larger payload.

Limitations

  • Maximum of 120,000 in-flight messages being processed via the consumer.
  • Max size per message 256KB.
  • Retention from 1 minute to 14 days.
  • Message content has to be of type String so either XML, JSON or text.
  • Standard queues have no transactional per second (TPS), so no limit.
  • FIFI queues default have a limit of 3,000 (batching).
  • Pay per API request and network usage is well.

Security

  • Encryption in flight using an HTTPS endpoint.
  • Can enable server side encryption via KMS. Will only encrypt the body.
  • IAM policy to allow usage of SQS.
  • SQS queue access policy, offers more fine grained control such as over IP.

--

--

Joshua Callis

Converted DevOps Engineer at oso.sh, Previously a Senior Software Engineer.