is "distributed event streaming platform"

was created by LinkedIn

Use-cases from conductor site

• Messaging systems
• Activity Tracking
• Gather metrics from many different locations, for example, IoT devices
• Application logs analysis
• De-coupling of system dependencies
• Integration with Big Data technologies like Spark, Flink, Storm, Hadoop
• Event-sourcing store You can find a list of use cases at https://kafka.apache.org/uses

message queues

mental model

cluster > broker > topic > partition :: consumer group > consumer

acts as buffer. difference from queues

tidbits -> kafka is used by seven out of ten banks in their technical systems.

explore

when is doubt check the Kafka UI by conductor which is more like an interface for command line with respect to all critical commands and parameters. they also use same defaults

when i talk to many full stack developers i see that they can hardly draw the similarities between kafka and redux

tips

1. better to use xfs instead of ext4
   1. forgot the reason dig-deeper
2. number of consumers is limited by the number of partitions. so the limit for parallelisation is number of partitions
3. increasing partitions is easy. decreasing is difficult
   1. get better intuition around this - dig-deeper

kafka metrics

https://docs.confluent.io/platform/current/kafka/monitoring.html#localtimems

Referenced in:

• consistent-hashing
• kafka-and-redux
• 16-kb
• group-memberships
• stop-the-world