Because the client requests pagination by lastEventId (a UUID), the server needs to remember every event forever in order to correctly catch up clients.
If instead the client paginated by lastEventTimestamp, then a server that for any reason no longer had a particular event UUID could at least start at the following one.
That’s why the article suggests using a uuid v6 which is time orderable. Or prefixing with an incrementing db id. So indeed, if you intend to delete events, you might want to make sure you have orderable ids of some sort.
What happens if you need to catch up? You keep calling in a loop with a new lastEventId?
What is the intention there though. Is this for social media type feeds, or is this meant for synchronising data (at the extreme for DB replication for example!).
What if anything is expected of the producer in terms of how long to store events?
I enjoy anything that drives down NIH, or something that an existing library could possibly support, or something that I could take to my next job (or could possibly hire for)
I believe cloud events are most common in Kafka-adjacent or event-driven architectures but I think they're used in some GCP serverless things, too
Use HTTP server-sent events instead. Those can keep the connection open so you don't have to poll to get real-time updates and they will also let you resume from the last entry you saw previously.
Yeah, but in real life, SSE error events are not robust, so you still have to do manual heartbeat messages and tear down and reestablish the connection when the user changes networks, etc. In the end, long-polling with batched events is not actually all that different from SSE with ping/pong heartbeats, and with long-polling, you get the benefit of normal load balancing and other standard HTTP things
If instead the client paginated by lastEventTimestamp, then a server that for any reason no longer had a particular event UUID could at least start at the following one.
For example, what inspired this over a typical paginated API that lets you sort old to new with an afterId parameter?
What is the intention there though. Is this for social media type feeds, or is this meant for synchronising data (at the extreme for DB replication for example!).
What if anything is expected of the producer in terms of how long to store events?
I believe cloud events are most common in Kafka-adjacent or event-driven architectures but I think they're used in some GCP serverless things, too
Use HTTP server-sent events instead. Those can keep the connection open so you don't have to poll to get real-time updates and they will also let you resume from the last entry you saw previously.
https://developer.mozilla.org/en-US/docs/Web/API/Server-sent...
I would also rather not have a handful of long-polling loops pollute the network tab.
(Details in the previous thread on HTTP Feeds: https://news.ycombinator.com/item?id=30908492 )