https://1shubham7.blog/tags/sre/ Recent content in SRE on Shubham Singh Hugo -- gohugo.io en-us https://1shubham7.blog/helm-features-for-sres/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/helm-features-for-sres/ Twenty-five Helm flags, template functions, chart features and tools that most daily users never touch, each with the operational problem it solves. Written after two years of writing and managing Helm packages in production. https://1shubham7.blog/agent-evals/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/agent-evals/ Normal code has tests; agents need evals. What an eval harness is, the vocabulary that goes with it, how I would build one in Go for an SRE agent working against Kubernetes, why pass^k matters more than pass@k for ops, the tools that exist (and the one OpenAI is shutting down), the ways a harness's numbers lie to you, and the habits that keep a suite useful. https://1shubham7.blog/helm-hooks/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/helm-hooks/ What Helm does with a hook step by step, the three things hooks do not do that everyone assumes they do, a migration Job that survives a rollback, and when an initContainer or an Argo CD sync wave is the better tool. Written after two years of writing and managing Helm packages in production. https://1shubham7.blog/log-collection-and-labels/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/log-collection-and-labels/ A container log line on disk contains no idea which pod wrote it. Everything you filter by later is constructed by the collector at collection time. What the file actually looks like, how collectors find it, where every label comes from, and why that design decides the health of your logging stack. https://1shubham7.blog/configmap-restart/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/configmap-restart/ Why a changed ConfigMap restarts nothing, the Helm checksum annotation done properly, the two mistakes that make it silently do nothing, and when an immutable ConfigMap, Reloader, or a reload sidecar is the better tool. Written after two years of writing and managing Helm packages in production. https://1shubham7.blog/rust-for-sres/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/rust-for-sres/ I learned enough Rust in a weekend to read, review and debug the CLIs, API clients and daemons that show up in infra work. This is the subset that mattered, the compiler errors you will actually hit, and the parts of the language I deliberately skipped. https://1shubham7.blog/system-one-models-jev/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/system-one-models-jev/ Most LLM calls in a backend are structured decisions dressed up as chat. TypeSafe's Jev answers only that kind of question: typed answers with probabilities, in one pass, with no text generation. What it is, how its confidence numbers work, what 'can't hallucinate' does and does not mean, the open-source alternatives that appeared within weeks, and where an SRE would put one. https://1shubham7.blog/layer-under-kubernetes/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/layer-under-kubernetes/ Which of my Kubernetes-shaped SRE instincts are about the abstraction and which are about the machine underneath, worked out against a hosting company that runs about 12,000 single-tenant VMs across ten cloud providers with no Kubernetes at all. https://1shubham7.blog/victoriametrics-stack/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/victoriametrics-stack/ VictoriaMetrics started as a remote storage backend for Prometheus and grew into a full monitoring stack. What Prometheus deliberately does not do, why two HA Prometheus replicas are not a cluster, how VictoriaMetrics splits Prometheus's jobs into components and keeps long-term data without object storage, how VictoriaLogs compares with Loki and VictoriaTraces with Tempo and Jaeger, what else the project ships, and when it is the right choice. https://1shubham7.blog/what-i-learned-reading-the-cyber-resilience-act/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/what-i-learned-reading-the-cyber-resilience-act/ I read the whole of Regulation (EU) 2024/2847, every article and every annex, and mapped KubeAid and LinuxAid against it for my employer. These are the things that surprised me, the mental model I ended up with, and what I would tell the next engineer who has to do the same reading. https://1shubham7.blog/linux-concepts/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/linux-concepts/ Processes and signals, namespaces and cgroups, memory, storage, networking, and identity: the mechanism behind each, the two or three ways it breaks under a pod, and the one check that confirms it. Written from on-call across 100+ production Kubernetes clusters. https://1shubham7.blog/cnp-antipatterns/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/cnp-antipatterns/ Lessons from writing CiliumNetworkPolicies for the KubeAid-addons chart - toServices port traps, labels Cilium ignores, toFQDNs without a DNS rule, default-deny surprises, and why every namespace needs exactly one default-deny. https://1shubham7.blog/loki-production-checklist/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/loki-production-checklist/ Deployment mode, object storage, label cardinality, multi-tenancy and auth, cross-cluster shipping, and the failures that are silent rather than loud. Written after moving a Loki install twice and debugging an object store that corrupted everything it was given. https://1shubham7.blog/experience/obmondo/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/experience/obmondo/ Outline Worked as an SRE, handling production systems and managing Kubernetes clusters and Linux servers across both internal and customer environments. As the youngest SRE at Obmondo, regularly took ownership of production alerts and incidents, performing root cause analysis and implementing long-term fixes. Led the rewrite of the most business-critical internal tool, improving its reliability, testing, and integration into GitOps workflows. Authored and improved a large part of the company’s ISMS, strengthening both the company’s and customers’ information security. https://1shubham7.blog/audit-logging/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/audit-logging/ A breakdown of how kube-apiserver audit events actually work - request stages, audit levels, writing policies, and how SREs tune them differently for prod vs dev clusters. https://1shubham7.blog/grafana-dashboards/ Mon, 01 Jan 0001 00:00:00 +0000 https://1shubham7.blog/grafana-dashboards/ Two Grafana dashboards, same node, same timestamp - one shows 62 GB used, other shows 10 GB. Both are correct. Here's the full story of how Linux memory actually works and which number you should actually be watching.