From monitoring to observability
open source options
Notes from https://future.a16z.com/the-case-for-developer-experience
Three pillars of observability
Observability is about building models of your software so you can build software more quickly
But design for complexity-exploring Tools — which is more about meeting developers where they are — means digesting larger parts of the rainforest that are the developer’s ecosystem.
Case in point: People talk about the pillars of observability as logs, metrics, and traces, instead of about goals like “understand your system behavior” or “catch breaking changes!”
Instead, we need to focus on more interoperability with existing dev tools, as well as on more incremental improvements (yes) that aren’t a so-called paradigm shift but that actually work with what exists.
developers don’t actually need to observe every aspect of their systems. Instead, they just want a prioritized view, one that gives them visibility into what matters. Prioritization matters far more than soundness, bugs, or comprehensiveness.
Observability stack
- LMA stack
- Logs
- Metrics
- Alerts
- Log aggregation
- Loki - Prometheus for logs
- ingests logs
- indexes
- query language(LogQL) similar to prometheus(PromQL)
- integration with grafana
- Loki - Prometheus for logs
- Prom tail
- Agent that is installed in the source.
- Log parsing and enriching
- Collects logs and pushes them to Loki
- Prometheus
- What Loki is for logs, prometheus is for metrics
Further reading
- https://twitter.com/hibaymj/status/1423426718799470595
- https://www.honeycomb.io/wp-content/uploads/2018/07/Honeycomb-Guide-Achieving-Observability-v1.pdf
- https://dl.acm.org/doi/10.1145/3411506.3417602
- https://shopify.dev/changelog
- https://medium.com/@gerstenzang/developer-tools-why-it-s-hard-to-build-a-big-business-423436993f1c
- https://www.akitasoftware.com/blog-posts/why-arent-there-more-programming-languages-startups
Referenced in:
All notes