https://1shubham7.blog/tags/observability/ Recent content in Observability on Shubham Singh Hugo -- gohugo.io en-us https://1shubham7.blog/log-collection-and-labels/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/log-collection-and-labels/ A container log line on disk contains no idea which pod wrote it. Everything you filter by later is constructed by the collector at collection time. What the file actually looks like, how collectors find it, where every label comes from, and why that design decides the health of your logging stack. https://1shubham7.blog/victoriametrics-stack/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/victoriametrics-stack/ VictoriaMetrics started as a remote storage backend for Prometheus and grew into a full monitoring stack. What Prometheus deliberately does not do, why two HA Prometheus replicas are not a cluster, how VictoriaMetrics splits Prometheus's jobs into components and keeps long-term data without object storage, how VictoriaLogs compares with Loki and VictoriaTraces with Tempo and Jaeger, what else the project ships, and when it is the right choice. https://1shubham7.blog/loki-production-checklist/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/loki-production-checklist/ Deployment mode, object storage, label cardinality, multi-tenancy and auth, cross-cluster shipping, and the failures that are silent rather than loud. Written after moving a Loki install twice and debugging an object store that corrupted everything it was given. https://1shubham7.blog/experience/obmondo/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/experience/obmondo/ Outline Worked as an SRE, handling production systems and managing Kubernetes clusters and Linux servers across both internal and customer environments. As the youngest SRE at Obmondo, regularly took ownership of production alerts and incidents, performing root cause analysis and implementing long-term fixes. Led the rewrite of the most business-critical internal tool, improving its reliability, testing, and integration into GitOps workflows. Authored and improved a large part of the company’s ISMS, strengthening both the company’s and customers’ information security. https://1shubham7.blog/grafana-dashboards/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/grafana-dashboards/ Two Grafana dashboards, same node, same timestamp - one shows 62 GB used, other shows 10 GB. Both are correct. Here's the full story of how Linux memory actually works and which number you should actually be watching.