Log collection: Fluent Bit vs Fluentd · Prasanta Kumar Behera

Choose and run the collector: Fluent Bit vs Fluentd vs Logstash (and why EFK with Fluent Bit suits small and medium clusters), Fluent Bit's pipeline model, a production DaemonSet configuration, and the buffering and retry settings that decide whether logs are delayed or silently lost.

The collectors

	Fluent Bit	Fluentd	Logstash
Language	C	Ruby (+ C)	JVM
Footprint	Very small	Moderate	Large
Plugins	Many, growing	Very many	Very many
Typical role	Per-node collector	Aggregator / complex routing	Central processing (ELK)
Project	CNCF (graduated, under Fluentd)	CNCF graduated	Elastic

Why EFK (with Fluent Bit) rather than ELK on small and medium clusters: a per-node collector must be cheap, because it runs on every node. Fluent Bit collects, parses, enriches and ships directly to Elasticsearch. Add a central Fluentd or Logstash tier only when you need heavy transformation, fan-out to many destinations, or a buffer in front of Elasticsearch. (Elastic's own agents, Filebeat and Elastic Agent, are another valid choice.)

Fluent Bit is a bike courier: light, quick and cheap enough to have one in every street. Fluentd and Logstash are delivery vans: they carry and repackage much more, but you wouldn't park one in every street. Big cities often use bikes to collect parcels and a few vans to sort them at a depot.

Fluent Bit's model

[INPUT tail] → parsers → [FILTER kubernetes] → [FILTER …] → buffer (memory/filesystem) → [OUTPUT es]
      tag: kube.*                 match: kube.*                                         match: kube.*

Each record carries a tag; filters and outputs select records with match patterns.

A production DaemonSet config (YAML format)

service:
  flush: 1
  log_level: info
  http_server: on                         # metrics on :2020
  storage.path: /var/fluent-bit/state/flb-storage/
  storage.sync: normal
  storage.backlog.mem_limit: 50M

pipeline:
  inputs:
    - name: tail
      tag: kube.*
      path: /var/log/containers/*.log
      exclude_path: /var/log/containers/fluent-bit*
      multiline.parser: cri, docker
      db: /var/fluent-bit/state/flb_kube.db   # remember file positions across restarts
      mem_buf_limit: 50MB
      skip_long_lines: on
      storage.type: filesystem

  filters:
    - name: kubernetes
      match: kube.*
      merge_log: on                        # parse JSON messages into fields
      keep_log: off
      k8s-logging.parser: on               # honour fluentbit.io/parser annotations
      k8s-logging.exclude: on              # honour fluentbit.io/exclude annotations

  outputs:
    - name: es
      match: kube.*
      host: logs-es-http.logging.svc
      port: 9200
      tls: on
      tls.verify: on
      tls.ca_file: /es-ca/ca.crt
      http_user: fluent-bit
      http_passwd: ${ES_PASSWORD}
      index: logs-k8s-default                # a data stream (template with data_stream: {})
      suppress_type_name: on
      retry_limit: no_limits
      storage.total_limit_size: 5G

Install with the fluent/fluent-bit Helm chart (repo https://fluent.github.io/helm-charts), which creates the DaemonSet, RBAC (the kubernetes filter reads pod metadata) and hostPath mounts. The chart's default config uses the classic .conf format; both formats express the same settings, and newer releases favour YAML.
Mount /var/fluent-bit/state from the host (hostPath) so buffers and file positions survive pod restarts.
Check the es output options for your Elasticsearch version (for data streams, writes must use the create operation, which is the default in current Fluent Bit versions).

Buffering, back-pressure and loss

Setting	Effect
`mem_buf_limit` (input)	Pauses the input when memory buffer is full
`storage.type: filesystem`	Buffers chunks on disk: survives restarts, absorbs outages
`storage.total_limit_size` (output)	Caps disk use; oldest chunks are dropped when full
`retry_limit` (output)	Default 1; set `no_limits` or a large number
kubelet log rotation	If Fluent Bit is paused longer than files live, lines are gone

Monitor Fluent Bit's metrics (fluentbit_output_retries_total, fluentbit_output_retries_failed_total, fluentbit_output_dropped_records_total, fluentbit_input_records_total) and alert on drops and failed retries.

Try it: collect and break

Install Fluent Bit with the Helm chart on the kind cluster from lesson 03, pointing at your ECK cluster (mount the CA Secret, create a fluent-bit user with a role that can write to logs-*).
Run the logger pod from lesson 01 and find its lines in Elasticsearch (GET logs-k8s-default/_search?q=hello).
Scale the Elasticsearch cluster to 0 data nodes (or block it with a NetworkPolicy) for 5 minutes; watch retries in Fluent Bit metrics; restore and check whether lines are missing.
Repeat with retry_limit: no_limits and filesystem storage; compare.
Add a stdout output temporarily and look at a fully enriched record.

Going deeper: collector architecture

For very high volumes or many destinations, put Kafka (or a Fluentd/Logstash tier) between collectors and Elasticsearch, so storage outages don't back up into nodes.
Collect node and system logs too (journald input for kubelet/containerd), and Kubernetes events (via an event exporter), into separate data streams.
Keep Fluent Bit's own resource limits realistic; an OOM-killed collector loses its memory buffer.
OpenTelemetry Collector can also collect logs (filelog receiver) and is worth considering if you're standardising on OTel (see Observability with OpenTelemetry).

Recap

Fluent Bit as the light per-node collector; Fluentd/Logstash/Kafka as optional central tiers. That's the EFK-over-ELK argument.
Pipeline: input → parser → filter → buffer → output, routed by tags.
Production config: tail with db and CRI multiline, the kubernetes filter, TLS and a least-privilege user.
Avoid silent loss: filesystem buffering, retry_limit no_limits, size caps, and metrics/alerts on drops.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.