Reworking my log monitoring

I previously built a log monitoring system that I've been using to review logs on my servers. It has helped me keep a better eye on logs and I've caught a few issues through it.

One thing I wasn't happy with was that it required all my servers to access a central server in order to ship their logs. Over time I disabled access from some servers I judged less critical because of this, which meant I lost the ability to see their logs.

I decided to rework the system so that none of the servers needed access to any of the others.

Each server now gathers and filters its logs and then publishes a message to a GCP Pub/Sub topic. The Pub/Sub message triggers a Cloud Function that sends me an email. The email contains a summary of interesting logs, generated by each server independently.

There's still a centralisation point, Pub/Sub, but my concern was access between my servers, and this setup solves that. None of the servers need access to anything except the ability to publish to Pub/Sub, and I'm much less concerned about that access.

It might seem strange to involve Pub/Sub. The reason for it is that I didn't want to give SMTP credentials to the servers. Instead, only the Cloud Function has the credentials, and it's small and self contained.

In effect this isolates each server, using Pub/Sub as its only way to contact me.

As well, I'm trying to reduce the responsibilities of my servers. With this new system, the previously central server's duties are much reduced. It doesn't need to run the HTTP daemon that was receiving the logs, nor does it need the PostgreSQL database storing them. The system is much simpler having each piece operate independently.