How I Set Up Grafana Email Alerts with Gmail, Cloudflare, and Kubernetes
I've been using Grafana in my homelab for a while to monitor things — dashboards for metrics, logs for troubleshooting. However, I had not yet set up any alerts. Mainly because I didn't have a good place to send them. I wanted to make sure alerts wouldn't go unnoticed and Email was the obvious choice — simple, reliable, and accessible from anywhere. But as usual in a homelab setup, the question became: how can I self-host?
The Self-Hosted Mail Server Rabbit Hole
Initially, I explored running my own mail server. There are some great all-in-one solutions like Mailu and Mailcow, and they can run in Docker or, in some cases, Kubernetes. But once I got into the details — besides that it wouldn't work behind a Cloudflare tunnel, there was reverse DNS, IP reputation, spam filtering, greylisting, and potential that my internet provider blocks the outgoing SMTP port to deal with — it was clear that for just sending and receiving a few alerts, it wasn’t worth the operational overhead. So I continued looking for something better.
Discovering Cloudflare Email Routing
Cloudflare’s Email Routing feature was a game changer. It lets you forward any email sent to your domain to a personal inbox like Gmail or Outlook — no mail server required.
Setup was super easy. I added a forwarding address to my Gmail, and Cloudflare showed which MX
and TXT
records I needed to add to my DNS configuration. Since my domain is already managed through Cloudflare, it was a one-click apply — and emails started arriving almost instantly. I didn’t need to expose ports, run postfix, or configure spam filters. Inbound mail? Solved.
Outbound: Sending with Gmail’s SMTP Relay
With inbound mail handled, next up was sending alerts from Grafana via email.
Rather than setting up a dedicated mail-sending service, I configured Grafana to send via Gmail's SMTP server. All it required was:
- An app-specific password (since I use 2FA)
- SMTP config pointed at
smtp.gmail.com:587
To keep everything secure and Git-friendly, I stored credentials in Kubernetes Secrets:
apiVersion: v1
kind: Secret
metadata:
name: grafana-smtp
namespace: monitoring
type: Opaque
stringData:
user: [email protected]
password: app-specific-password
Then I referenced them in values.yaml
when deploying Grafana via Helm:
grafana:
grafana.ini:
smtp:
enabled: true
host: smtp.gmail.com:587
from_name: Grafana Alerts
smtp:
existingSecret: grafana-smtp
userKey: user
passwordKey: password
I also templated the from_address
using a second secret, ensuring nothing sensitive ended up in version control. Then to make sure I didn't accidently forget to add it when updating the deployment, I added a short bash script to deploy the chart with.
#!/bin/bash
NAMESPACE="monitoring"
RELEASE_NAME="grafana"
CHART_NAME="grafana/grafana"
EMAIL_SECRET_NAME="grafana-email"
EMAIL_KEY="from_address"
# Pull from_address from the Kubernetes Secret
FROM_EMAIL=$(kubectl get secret "$EMAIL_SECRET_NAME" \
-n "$NAMESPACE" \
-o "jsonpath={.data.${EMAIL_KEY}}" | base64 --decode)
if [ -z "$FROM_EMAIL" ]; then
echo "❌ Could not retrieve from_address from secret '$EMAIL_SECRET_NAME'"
exit 1
fi
echo "📧 Using from_address: $FROM_EMAIL"
helm upgrade --install "$RELEASE_NAME" "$CHART_NAME" \
-n "$NAMESPACE" \
-f values.yaml \
--set "grafana.grafana.ini.smtp.from_address=${FROM_EMAIL}"
After setting that up, I sent a test alert in Grafana and... nothing. The odd thing was I did see that something happened in Cloudflare's Email Routing dashboard. So what was going on?
After checking my Sent messages in Gmail, I did see it there. However, it seemed like it just stayed there, rather than going to my Inbox. That is definitely something I needed to happen for this to be useful.
After some Googling, I realized I needed to add the custom address to Gmail under "Send mail as" and verify it. After recieving the verification email and clicking the verification link, I tested again and it showed up in my Inbox!
In addition, I made sure to set up a filter to avoid marking it as spam and:
- Categorize it as "Primary"
- Apply an "Alert" label
- Mark it as important
Conclusion
I started out thinking I’d need to run my own mail server, but Cloudflare and Gmail gave me exactly what I needed with a fraction of the hassle. With this setup:
- ✅ I can receive email at a custom domain without hosting mail
- ✅ Grafana alerts send reliably via Gmail SMTP
- ✅ All secrets are stored securely in Kubernetes
- ✅ Gmail filters keep alerts visible and organized
For a homelab setup, this strikes the perfect balance of control, simplicity, and reliability. Now I just need to configure more alerts to take full advantage of it.