Open source data collector for Unified Logging - Fluentd

Fluentd is an open source data collector for unified logging layer. It allows you to unify data collection and consumption for a better use and understanding of data.

Fluentd decouples data sources from backend systems by providing a unified logging layer in between. It is Apache 2.0 Licensed, fully open source software. Fluentd treats logs as JSON, a popular machine-readable format. It is written primarily in C with a thin-Ruby wrapper that gives users flexibility.

Fluentd is an open source log management tool supported by the CNCF that unifies your data collection in a language- and platform-agnostic manner. It brings together data from your databases, system logs, and application events, filters out the noise, and then structures that data so it can be easily fed out to multiple destinations. Through its flexible plugin architecture, Fluentd works with hundreds of different services, from commercial products like Splunk to open source tools like ElasticSearch or MongoDB. Prized for microservices architecture, Fluentd is also an excellent choice for legacy and monolithic applications. Its reduced footprint sibling Fluent Bit is even applicable for the Internet of Things.

 

ref:

Fluentd - https://www.fluentd.org/

Fluentd overview - https://docs.fluentd.org/quickstart 

Fluentd github - https://github.com/fluent/fluentd

Fluentd community - https://www.fluentd.org/community

Fluentd wiki - https://en.wikipedia.org/wiki/Fluentd

Fluentd as part of CNCF - 

    https://landscape.cncf.io/selected=fluentd

    https://epsagon.com/tools/cncf-tools-overview-fluentd-unified-logging-layer/

Aggregating Application Logs from Kubernetes Clusters using Fluentd to Log Intelligence - 

    https://medium.com/@bahubalishetti/aggregating-application-logs-from-kubernetes-clusters-using-fluentd-to-log-intelligence-91da5f536692

    https://medium.com/kubernetes-tutorials/cluster-level-logging-in-kubernetes-with-fluentd-e59aa2b6093a

Analyzing logs in real time using Fluentd and BigQuery - https://cloud.google.com/solutions/real-time/fluentd-bigquery