一、创建configmap配置文件
注:filebeat6以上版本需要将prospectors改为inputs,paths下指定的nginx-ingress日志路径匹配模式以及hosts指定的kafka地址需要根据实际修改,document_type和topic需要是kafka中存在的
[[email protected] ~]# cat >> filebeat-config.yaml <<EOF
> apiVersion: v1
> data:
> filebeat.yml: |
> filebeat.prospectors:
> – input_type: log
> paths:
> – “/var/log/containers/nginx-ingress-ingress-nginx-controller-*_ingress-nginx_controller-*.log”
> symlinks: true
> json.message_key: log
> json.keys_under_root: true
> json.add_error_key: true
> multiline.pattern: ‘^/s’
> multiline.match: after
> document_type: k8s-logs
> fields:
> POD_NAME: ‘nginx-ingress’
> fields_under_root: true
> exclude_lines: [‘/.(xml|gif|jpg|jpeg|png|bmp|swf|woff|woff2|ttf|js|css|svg|ico)’]
>
>
> output.kafka:
> hosts: [“kafka-service:9092”]
> topic: “k8s-logs”
> required_acks: 1
> kind: ConfigMap
> metadata:
> name: filebeat-configmap
> namespace: middleware
> EOF
二、创建configmap
[[email protected] ~]# kubectl apply -f filebeat-config.yaml
三、创建filebeat的daemonset配置文件
[[email protected] ~]# cat >> filebeat.yaml <<EOF > apiVersion: apps/v1 > kind: DaemonSet > metadata: > labels: > app: filebeat > name: filebeat > namespace: middleware > spec: > revisionHistoryLimit: 10 > selector: > matchLabels: > app: filebeat > template: > metadata: > labels: > app: filebeat > spec: > containers: > - image: elastic/filebeat:5.6.14 > imagePullPolicy: Always > name: filebeat > resources: > limits: > cpu: 200m > memory: 200Mi > requests: > cpu: 100m > memory: 100Mi > terminationMessagePath: /dev/termination-log > terminationMessagePolicy: File > volumeMounts: > - mountPath: /etc/filebeat/ > name: config > readOnly: true > - mountPath: /var/log/containers > name: varlog > - mountPath: /var/log/pods > name: varlogpods > readOnly: true > - mountPath: /var/lib/docker/containers > name: varlibdockercontainers > - mountPath: /var/host/log > name: volume-1621230243442 > dnsPolicy: ClusterFirst > imagePullSecrets: > - name: gaqmt > restartPolicy: Always > schedulerName: default-scheduler > securityContext: {} > terminationGracePeriodSeconds: 30 > tolerations: > - effect: NoSchedule > key: node-role.kubernetes.io/master > operator: Exists > volumes: > - configMap: > defaultMode: 420 > items: > - key: filebeat.yml > path: filebeat.yml > name: filebeat-configmap > name: config > - hostPath: > path: /var/log/containers > type: "" > name: varlog > - hostPath: > path: /var/log/pods > type: "" > name: varlogpods > - hostPath: > path: /var/lib/docker/containers > type: "" > name: varlibdockercontainers > - hostPath: > path: /var/log > type: "" > name: volume-1621230243442 > updateStrategy: > rollingUpdate: > maxUnavailable: 1 > type: RollingUpdate > EOF
四、创建filebeat的pod
[[email protected] ~]# kubectl apply -f filebeat.yaml
原创文章,作者:ItWorker,如若转载,请注明出处:https://blog.ytso.com/tech/aiops/281076.html