#EDA Enough already with ‘Event Streaming’ **
Anyone watching the current (streaming?) video series of the 2022 ‘EDA Summit’ is likely to once again be shocked by the number of EDA pioneers making interminable references to ‘Event Streaming’. Why shocked? Because Events are NEVER ‘Streamed’, and anyone claiming to understand EDA probably ought to understand that fairly fundamental point.
The Oxford Dictionary defines ‘Streaming’ as: “A method of transmitting or receiving data (especially video and audio material) … as a steady, continuous flow, allowing playback to start while the rest of the data is still being received”. Events, you are probably aware, are NEVER split across separate payloads. Indeed, on major platforms such as AWS where event payloads are limited to a maximum of 256 KB, even then I have never seen an individual event payload that came close to exceeding this limit.
Some might argue that if events ever did pass the 256 KB limit set by AWS, then at that point in time, those events will indeed need to be ‘Streamed’: to be split across numerous payloads? In fact, at least in the case of AWS (and my space is limited), even those payloads aren’t split: they are in no way ‘Streamed’. The full Event Payload is instead delivered to an S3 Bucket, and the trimmed ‘Event’ is sent as normal, with the exception of it containing a link to the full Event Payload in S3. As such, even in the example of extremely large event payloads on AWS, those payloads are never ‘Streamed’ (no one sends videos as events).
So does the term ‘Event Streaming’ ever make sense? In fact yes, but only when followed by ‘Analytics’: ‘Event Streaming Analytics’. In the case of Event ‘Streaming Analytics’, events are analyzed as a — logical — ‘Stream’: within the context of a rolling-window of time. (Event) Streaming Analytics almost always relates to IoT, where the last — rolling — 30 minutes of events, for example, must be compared to each other to detect discrepancies. But once again, each individual Payload delivered within the rolling window under analysis, was received all-at-once, in it’s integrality: fairly obvious given the micro-size of IoT (MQTT) payloads.
This might all seem quite trivial, like normal marketing blah blah, but I believe that those pundits within the EDA community making endless references to ‘Event Streaming’ — and here I think especially of the those pushing Kafka (which as far as I am aware has no pre-packaged ‘Streaming Analytics’ capabilities: only Java/Scala libraries inviting you to code your own) — are in fact doing the community a disservice by adding unnecessary (and factually incorrect) confusion. The next time they receive 10 spam emails in the space of a minute, they should ask themselves if those emails weren’t in fact ‘Streamed’ to them? If MS Outlook isn’t in the end an ‘Email Streaming’ client?
