How do I resume reading after a disconnect without missing messages?
Keep the cursor of the last message you handled, and ask for messages after it. CyberPlex stores messages durably, so a reader that was offline gets everything newer than its cursor when it returns, in order, even after the service restarted.
How it works
Every message has a cursor, and every read returns one. Pass the one you last handled:
curl -s "$CYBERPLEX_URL/v1/channels/tasks/messages?cursor=v1.42&limit=100" -H "Authorization: Bearer $TOKEN"
cursor = load_saved_cursor() or the channel's head cursor
loop:
page = GET messages?cursor=<cursor>&limit=100
for each message in page.publications:
handle it
save message.cursor # advance after each message
if page.hasMore: ask again from the last cursor
Treat the cursor as an opaque string: store it and send it back untouched, and never parse or compare cursors yourself.
Exactly where the guarantee ends
- There are no acknowledgements. Your cursor is your delivery state, and you keep it.
- A crash can repeat one message. If your process dies after handling a message but before saving the cursor, that message is delivered again on restart. Make your handlers idempotent, for example by recording the message id you acted on.
- History is bounded. CyberPlex keeps a tenant's newest 1,000 messages and nothing older than 7 days. If you were away longer, or the channel moved on by more than that, the response says
"truncated": true: you missed some messages, and you carry on from where the response resumes rather than being silently skipped. - Order is per channel. Messages on one channel arrive in order. There is no ordering across channels.
When not to use CyberPlex
- You cannot lose a message, ever. Retention is capped by count and age, and an offline reader that exceeds either loses the oldest messages (you are told, but you cannot recover them). If a message must be kept until it is processed, use a durable queue with acknowledgements, or store the work in your own database and use CyberPlex only to notify.
- You need exactly-once processing. You get at-least-once delivery with a cursor you save after handling, and effectively-once only if your handler is idempotent.
- You need to replay a long history. Seven days and 1,000 messages are the limits, so it is not an event log or an archive.
- Several workers should share one stream and split the messages. Each reader tracks its own cursor, so there is no automatic sharing of work between readers.