01 Flume系列(一)安装配置详解大数据

01 Flume系列(一)安装配置

Flume(http://flume.apache.org/) is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.

Flume 是 Cloudera 提供的一个高可用的,高可靠的,分布式的海量日志采集、聚合和传输的系统。Flume 是基于流数据的架构,用于在线日志分析。

Flume 架构

一、Flume 安装及配置

Flume 启动教程:http://flume.apache.org/FlumeUserGuide.html#starting-an-agent

(1) Flume 下载

tar -zxvf apache-flume-1.8.0-bin

(2) Flume 配置

vi conf/example.conf 
 
# 声明三种组件 
a1.sources = r1 
a1.sinks = k1 
a1.channels = c1 
 
# 定义source信息 
a1.sources.r1.type = netcat 
a1.sources.r1.bind = localhost 
a1.sources.r1.port = 44444 
 
# 定义sinks信息 
a1.sinks.k1.type = logger 
 
# 定义channels信息 
a1.channels.c1.type = memory 
 
# 绑定在一起 
a1.sources.r1.channels = c1 
a1.sinks.k1.channel = c1

(3) Flume 启动

bin/flume-ng agent -f conf/example.conf -n a1 -Dflume.root.logger=INFO,console

补充:netcat 简介

yum -y install netcat 
# nc 监听 
nc -lk localhost 8888 
# nc 登陆 
nc localhost 8888

每天用心记录一点点。内容也许不重要,但习惯很重要!

原创文章,作者:ItWorker,如若转载,请注明出处:https://blog.ytso.com/9828.html

(0)
上一篇 2021年7月19日
下一篇 2021年7月19日

相关推荐

发表回复

登录后才能评论